Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a constrained Markov decision problem (CMDP), and by using a Lagrangian relaxation we decompose it into single-client problems. Further, the optimal policy takes a threshold form in the video buffer length. We then derive a natural policy gradient (NPG) based constrained reinforcement learning (CRL) algorithm using the structure of our problem, and show that it converges to the globally optimal policy. We then develop a simulation environment for training, and a real-world intelligent controller attached to a WiFi access point for evaluation. We demonstrate using youtube media streaming experiments that our policy can increase the user quality of experience by over 30%. Furthermore, we show that the structured learning is fast, and can be easily deployed, taking only about 15μs to execute.

We consider a large-scale service marketplace with numerous servers that scale with the job arrival rate. Jobs arrive with private valuations representing their willingness to pay. In a centralized system, jobs are matched to available servers, and prices are set in a centralized manner to maximize revenue. We investigate whether similar scalability can be achieved in a distributed marketplace where jobs are randomly matched to servers, which set their own prices based on job valuations and system occupancy. Our results show that matching a job to a single randomly selected server leads to higher blocking, resulting in lower throughput and reduced revenue. We then examine matching jobs to two servers, which compete to provide service if unoccupied. We demonstrate the existence of a mean field equilibrium (MFE) in this setup, where servers strategically respond to competitors’ prices. We characterize the MFE and show that this two-server choice ensures lower blocking probabilities and higher system revenue. Our findings are validated through simulations illustrating a variety of operating scenarios.

Pre-trained transformers exhibit the capability of adapting to new tasks through in-context learning (ICL), where they efficiently utilize a limited set of prompts without explicit model optimization. The canonical communication problem of estimating transmitted symbols from received observations can be modeled as an in-context learning problem: Received observations are a noisy function of transmitted symbols, and this function can be represented by an unknown parameter whose statistics depend on an unknown latent context. This problem, which we term in-context estimation (ICE), has significantly greater complexity than the extensively studied linear regression problem. The optimal solution to the ICE problem is a non-linear function of the underlying context. In this paper, we prove that, for a subclass of such problems, a single-layer softmax attention transformer (SAT) computes the optimal solution of the above estimation problem in the limit of large prompt length. We also prove that the optimal configuration of such a transformer is indeed the minimizer of the corresponding training loss. Further, we empirically demonstrate the proficiency of multi-layer transformers in efficiently solving broader in-context estimation problems. Through extensive simulations, we show that solving ICE problems using transformers significantly outperforms standard approaches. Moreover, just with a few context examples, it achieves the same performance as an estimator with perfect knowledge of the latent context.

We address the challenge of zeroth-order online convex optimization where the objective function's gradient exhibits sparsity, indicating that only a small number of dimensions possess non-zero gradients. Our aim is to leverage this sparsity to obtain useful estimates of the objective function's gradient even when the only information available is a limited number of function samples. Our motivation stems from the optimization of large-scale queueing networks that process time-sensitive jobs. Here, a job must be processed by potentially many queues in sequence to produce an output, and the service time at any queue is a function of the resources allocated to that queue. Since resources are costly, the end-to-end latency for jobs must be balanced with the overall cost of the resources used. While the number of queues is substantial, the latency function primarily reacts to resource changes in only a few, rendering the gradient sparse. We tackle this problem by introducing the Compressive Online Gradient Optimization framework which allows compressive sensing methods previously applied to stochastic optimization to achieve regret bounds with an optimal dependence on the time horizon without the full problem dimension appearing in the bound. For specific algorithms, we reduce the samples required per gradient estimate to scale with the gradient's sparsity factor rather than its full dimensionality. Numerical simulations and real-world microservices benchmarks demonstrate CONGO's superiority over gradient descent approaches that do not account for sparsity.

Restless multi-armed bandits (RMAB) has been widely used to model constrained sequential decision making problems, where the state of each restless arm evolves according to a Markov chain and each state transition generates a scalar reward. However, the success of RMAB crucially relies on the availability and quality of reward signals. Unfortunately, specifying an exact reward function in practice can be challenging and even infeasible. In this paper, we introduce Pref-RMAB, a new RMAB model in the presence of preference signals, where the decision maker only observes pairwise preference feedback rather than scalar reward from the activated arms at each decision epoch. Preference feedback, however, arguably contains less information than the scalar reward, which makes Pref-RMAB seemingly more difficult. To address this challenge, we present a direct online preference learning (DOPL) algorithm for Pref-RMAB to efficiently explore the unknown environments, adaptively collect preference data in an online manner, and directly leverage the preference feedback for decision-makings. We prove that DOPL yields a sublinear regret. To our best knowledge, this is the first algorithm to ensure $\tilde{\mathcal{O}}(\sqrt{T\ln T})$ regret for RMAB with preference feedback. Experimental results further demonstrate the effectiveness of DOPL.

With the proliferation of short form high quality video content, it has become increasing important to find light weight and efficient edge caching algorithms that can quickly adapt to changing trends. In this context we study an online caching problem where a set of users are connected to a set of caches. The users request files from these caches over a time horizon. These requests arrive sequentially, the sequence of requests are divided into tasks that have a certain degree of similarity. This similarity is leveraged so that we may learn the best policy for a new task using a very small number of sequential requests. We characterize the task averaged regret incurred in this setting, showing an improvement of$D/D^{*}$where D is the diameter of the set of cache configurations and$D^{*}$is a measure of task similarity. We provide the same theoretical guarantees under both a distributed and smoothed setting. Further, we validate our algorithm on trace based data as well as on synthetic data sets. In the trace based data sets we do not assume any inherent task structure or estimate of$D^{*}$. These simulations show not only fast adaptation to new incoming tasks but also improved performance in highly non-stationary request settings.

We study the optimal control of multiple video streams over a wireless downlink from a base-transceiver-station (BTS)/access point to N end-devices (EDs). The BTS sends video packets to each ED under a joint transmission energy constraint, the EDs choose when to play out the received packets, and the collective goal is to provide a high Quality-of-Experience (QoE) to the clients/end-users. All EDs send feedback about their states and actions to the BTS which reaches it after a fixed deterministic delay. We analyze this team problem with delayed feedback as a cooperative Multi-Agent Constrained Partially Observable Markov Decision Process (MA-C-POMDP).First, using a recently established strong duality result for MAC-POMDPs, the original problem is decomposed into N independent unconstrained transmitter-receiver (two-agent) problems— all sharing a Lagrange multiplier (that also needs to be optimized for optimal control). Thereafter, the common information (CI) approach and the formalism of approximate information states (AISs) are used to guide the design of a neural-network based architecture for learning-based multi-agent control in a single unconstrained transmitter-receiver problem. Finally, simulations on a single transmitter-receiver pair with a stylized QoE model are performed to highlight the advantage of delay-aware two-agent coordination over the transmitter choosing both transmission and play-out actions (perceiving the delayed state of the receiver as its current state).

This work presents Windex, a novel light weight whittle index network-driven realtime scheduler for scalable service guarantees in NextG cellular networks. Windex addresses the resource allocation challenge in NextG cellular radio access networks (RAN), where resources must be shared among diverse user applications, each requiring guarantees on throughput and service regularity, taking into account service guarantees, channel quality, and system load. Implemented in a real time intelligent controller (RIC), and evaluating across standardized 3GPP service classes, we demonstrate the least service violations compared to state-of-the-art systems using over-the-air channel traces on a 5G testbed.

This work presents SPARC (Spatio-Temporal Adaptive Resource Control), a novel approach for multi-site spectrum management in NextG cellular networks. SPARC addresses the challenge of limited licensed spectrum in dynamic environments. We leverage the O-RAN architecture to develop a multi-timescale RAN Intelligent Controller (RIC) framework, featuring an xApp for near-real-time interference detection and localization, and a μApp for real-time intelligent resource allocation. By utilizing base stations as spectrum sensors, SPARC enables efficient and fine-grained dynamic resource allocation across multiple sites, enhancing signal-to-noise ratio (SNR) by up to 7dB, spectral efficiency by up to 15%, and overall system throughput by up to 20%.

The 5G network promised transformative services across various industries, yet its integration has mostly been limited to existing 4G services like IMS-based multimedia and IoT. This paper identifies two key reasons for this underutilization: first, the stringent, multi-dimensional requirements of next-generation verticals like AR/VR, mobile gaming, and robotics, and the limitations of traditional Quality of Service (QoS) approaches in meeting these needs. We argue for a shift towards Quality of Experience (QoE), which better captures user perception and instantaneous application state. To meet stringent QoE demands and optimize network utilization, we emphasize the importance of application state and context awareness within the network. As a solution, we propose AppNet, a novel framework that integrates application awareness into the networking stack via RAN Intelligent Controllers (RICs) of the Open-RAN platform, enabling dynamic QoS adjustments based on application context. This paper highlights 5G private networks as an ideal testing ground, focusing on multi-user scenarios to deliver optimal real-time interactive services at scale.

Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a constrained Markov decision problem (CMDP), and by using a Lagrangian relaxation we decompose it into single-client problems. Further, the optimal policy takes a threshold form in the video buffer length. We then derive a natural policy gradient (NPG) based constrained reinforcement learning (CRL) algorithm using the structure of our problem, and show that it converges to the globally optimal policy. We then develop a simulation environment for training, and a real-world intelligent controller attached to a WiFi access point for evaluation. We demonstrate using youtube media streaming experiments that our policy can increase the user quality of experience by over 30%. Furthermore, we show that the structured learning is fast, and can be easily deployed, taking only about 15μs to execute.

We consider the challenge of mitigating the generation of negative or toxic content by the Large Language Models (LLMs) in response to certain prompts. We propose integrating risk-averse principles into LLM fine-tuning to minimize the occurrence of harmful outputs, particularly rare but significant events. By optimizing the risk measure of Conditional Value at Risk (CVaR), our methodology trains LLMs to exhibit superior performance in avoiding toxic outputs while maintaining effectiveness in generative tasks. Empirical evaluations on sentiment modification and toxicity mitigation tasks demonstrate the efficacy of risk-averse reinforcement learning with human feedback (RLHF) in promoting a safer and more constructive online discourse environment.

We consider the problem of federated offline reinforcement learning (RL), a scenario under which distributed learning agents must collaboratively learn a high-quality control policy only using small pre-collected datasets generated according to different unknown behavior policies. Na\"{i}vely combining a standard offline RL approach with a standard federated learning approach to solve this problem can lead to poorly performing policies. In response, we develop the Federated Ensemble-Directed Offline Reinforcement Learning Algorithm (FEDORA), which distills the collective wisdom of the clients using an ensemble learning approach. We develop the FEDORA codebase to utilize distributed compute resources on a federated learning platform. We show that FEDORA significantly outperforms other approaches, including offline RL over the combined data pool, in various complex continuous control environments and real-world datasets. Finally, we demonstrate the performance of FEDORA in the real-world on a mobile robot. We provide our code and a video of our experiments at \url{https://github.com/DesikRengarajan/FEDORA}.

NextG cellular networks must support diverse applications, such as interactive media streaming or robot control that have strict requirements on throughput, latency and reliability. These requirements must be met via optimizing wireless resources by utilizing application layer information, such as media streaming stall counts or robot pose estimates, along with network information, such as channel qualities and backlogs.

We consider an ultra-dense wireless network with$N$channels and$M = N$devices. Messages with fresh information are generated at each device according to a random process and need to be transmitted to an access point. The value of a message decreases as it ages, so each device searches for an idle channel to transmit the message as soon as it can. However, each channel probing is associated with a fixed cost (energy), so a device needs to adapt its probing rate based on the “age” of the message. At each device, the design of the optimal probing strategy can be formulated as an infinite horizon Markov Decision Process (MDP) where the devices compete with each other to find idle channels. While it is natural to view the system as a Bayesian game, it is often intractable to analyze such a system. Thus, we use the Mean Field Game (MFG) approach to analyze the system in a large-system regime, where the number of devices is very large, to understand the structure of the problem and to find efficient probing strategies. We present an analysis based on the MFG perspective. We begin by characterizing the space of valid policies and use this to show the existence of a Mean Field Nash Equilibrium (MFNE) in a constrained set for any general increasing cost functions with diminishing rewards. Further we provide an algorithm for computing the equilibrium for any given device, and the corresponding age-dependent channel probing policy.

RAN Intelligent Control (RIC) has developed in parallel with Open Radio Access Networks (O-RAN) as a means of utilizing newly available interfaces. Focus has been largely on non-realtime (non-RT: > 1 sec) dealing with RAN management and offline training, and near-realtime (near-RT: 10 ms to 1 sec) dealing with UE load balancing and RAN configuration. We contend that the true power of RIC can be unleashed only with realtime (RT: < 100 μs) measurement, optimization, and control of RAN resources, corresponding to the cellular transmission time interval (TTI: 125 μs to 1 ms).

Safe reinforcement learning is extremely challenging--not only must the agent explore an unknown environment, it must do so while ensuring no safety constraint violations. We formulate this safe reinforcement learning (RL) problem using the framework of a finite-horizon Constrained Markov Decision Process (CMDP) with an unknown transition probability function, where we model the safety requirements as constraints on the expected cumulative costs that must be satisfied during all episodes of learning. We propose a model-based safe RL algorithm that we call Doubly Optimistic and Pessimistic Exploration (DOPE), and show that it achieves an objective regret $\tilde{O}(|\mathcal{S}|\sqrt{|\mathcal{A}| K})$ without violating the safety constraints during learning, where $|\mathcal{S}|$ is the number of states, $|\mathcal{A}|$ is the number of actions, and $K$ is the number of learning episodes. Our key idea is to combine a reward bonus for exploration (optimism) with a conservative constraint (pessimism), in addition to the standard optimistic model-based exploration. DOPE is not only able to improve the objective regret bound, but also shows a significant empirical performance improvement as compared to earlier optimism-pessimism approaches.

Meta reinforcement learning (Meta-RL) is an approach wherein the experience gained from solving a variety of tasks is distilled into a meta-policy. The meta-policy, when adapted over only a small (or just a single) number of steps, is able to perform near-optimally on a new, related task. However, a major challenge to adopting this approach to solve real-world problems is that they are often associated with sparse reward functions that only indicate whether a task is completed partially or fully. We consider the situation where some data, possibly generated by a sub-optimal agent, is available for each task. We then develop a class of algorithms entitled Enhanced Meta-RL via Demonstrations (EMRLD) that exploit this information---even if sub-optimal---to obtain guidance during training. We show how EMRLD jointly utilizes RL and supervised learning over the offline data to generate a meta-policy that demonstrates monotone performance improvements. We also develop a warm started variant called EMRLD-WS that is particularly efficient for sub-optimal demonstration data. Finally, we show that our EMRLD algorithms significantly outperform existing approaches in a variety of sparse reward environments, including that of a mobile robot.

The predominant use of wireless access networks is for media streaming applications. However, current access networks treat all packets identically, and lack the agility to determine which clients are most in need of service at a given time. Software reconfigurability of networking devices has seen wide adoption, and this in turn implies that agile control policies can be now instantiated on access networks. Exploiting such reconfigurability requires the design of a system that can enable a configuration, measure the impact on the application performance (Quality of Experience), and adaptively select a new configuration. Effectively, this feedback loop is a Markov Decision Process whose parameters are unknown. The goal of this work is to develop QFlow, a platform that instantiates this feedback loop, and instantiate a variety of control policies over it. We use the popular application of video streaming over YouTube as our use case. Our context is priority queueing, with the action space being that of determining which clients should be assigned to each queue at each decision period. We first develop policies based on model-based and model-free reinforcement learning. We then design an auction-based system under which clients place bids for priority service, as well as a more structured index-based policy. Through experiments, we show how these learning-based policies on QFlow are able to select the right clients for prioritization in a high-load scenario to outperform the best known solutions with over 25% improvement in QoE, and a perfect QoE score of 5 over 85% of the time.

Crucial performance metrics of a caching algorithm include its ability to quickly and accurately learn a popularity distribution of requests. However, a majority of work on analytical performance analysis focuses on hit probability after an asymptotically large time has elapsed. We consider an online learning viewpoint, and characterize the “regret” in terms of the finite time difference between the hits achieved by a candidate caching algorithm with respect to a genie-aided scheme that places the most popular items in the cache. We first consider the Full Observation regime wherein all requests are seen by the cache. We show that the Least Frequently Used (LFU) algorithm is able to achieve order optimal regret, which is matched by an efficient counting algorithm design that we call LFU-Lite. We then consider the Partial Observation regime wherein only requests for items currently cached are seen by the cache, making it similar to an online learning problem related to the multi-armed bandit problem. We show how approaching this “caching bandit” using traditional approaches yields either high complexity or regret, but a simple algorithm design that exploits the structure of the distribution can ensure order optimal regret. We conclude by illustrating our insights using numerical simulations.

Many physical systems have underlying safety considerations that require that the policy employed ensures the satisfaction of a set of constraints. The analytical formulation usually takes the form of a Constrained Markov Decision Process (CMDP). We focus on the case where the CMDP is unknown, and RL algorithms obtain samples to discover the model and compute an optimal constrained policy. Our goal is to characterize the relationship between safety constraints and the number of samples needed to ensure a desired level of accuracy---both objective maximization and constraint satisfaction---in a PAC sense. We explore two classes of RL algorithms, namely, (i) a generative model based approach, wherein samples are taken initially to estimate a model, and (ii) an online approach, wherein the model is updated as samples are obtained. Our main finding is that compared to the best known bounds of the unconstrained regime, the sample complexity of constrained RL algorithms are increased by a factor that is logarithmic in the number of constraints, which suggests that the approach may be easily utilized in real systems.

Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications.

In many real-world reinforcement learning (RL) problems, in addition to maximizing the objective, the learning agent has to maintain some necessary safety constraints. We formulate the problem of learning a safe policy as an infinite-horizon discounted Constrained Markov Decision Process (CMDP) with an unknown transition probability matrix, where the safety requirements are modeled as constraints on expected cumulative costs. We propose two model-based constrained reinforcement learning (CRL) algorithms for learning a safe policy, namely, (i) GM-CRL algorithm, where the algorithm has access to a generative model, and (ii) UC-CRL algorithm, where the algorithm learns the model using an upper confidence style online exploration method. We characterize the sample complexity of these algorithms, i.e., the the number of samples needed to ensure a desired level of accuracy with high probability, both with respect to objective maximization and constraint satisfaction.

We consider an ultra-dense wireless network with N channels and M = N devices. Messages with fresh information are generated at each device according to a random process and need to be transmitted to an access point. The value of a message decreases as it ages, so each device searches for an idle channel to transmit the message as soon as it can. However, each channel probing is associated with a fixed cost (energy), so a device needs to adapt its probing rate based on the "age" of the message. At each device, the design of the optimal probing strategy can be formulated as an infinite horizon Markov Decision Process (MDP) where the devices compete with each other to find idle channels. While it is natural to view the system as a Bayesian game, it is often intractable to analyze such a system. Thus, we use the Mean Field Game (MFG) approach to analyze the system in a large-system regime, where the number of devices is very large, to understand the structure of the problem and to find efficient probing strategies. We present an analysis based on the MFG perspective. We begin by characterizing the space of valid policies and use this to show the existence of a Mean Field Nash Equilibrium (MFNE) in a constrained set for any general increasing cost functions with diminishing rewards. Further we provide an algorithm for computing the equilibrium for any given device, and the corresponding age-dependent channel probing policy.

Whittle index policy is a powerful tool to obtain asymptotically optimal solutions for the notoriously intractable problem of restless bandits. However, finding the Whittle indices remains a difficult problem for many practical restless bandits with convoluted transition kernels. This paper proposes NeurWIN, a neural Whittle index network that seeks to learn the Whittle indices for any restless bandits by leveraging mathematical properties of the Whittle indices. We show that a neural network that produces the Whittle index is also one that produces the optimal control for a set of Markov decision problems. This property motivates using deep reinforcement learning for the training of NeurWIN. We demonstrate the utility of NeurWIN by evaluating its performance for three recently studied restless bandit problems.Our experiment results show that the performance of NeurWIN is significantly better than other RL algorithms.

The ability of a P2P network to scale its throughput up in proportion to the arrival rate of peers has recently been shown to be crucially dependent on the chunk sharing policy employed. Some policies can result in low frequencies of a particular chunk, known as the missing chunk syndrome, which can dramatically reduce throughput and lead to instability of the system. For instance, commonly used policies that nominally “boost” the sharing of infrequent chunks such as the well-known rarest-first algorithm have been shown to be unstable. We take a complementary viewpoint, and instead consider a policy that simply prevents the sharing of the most frequent chunk(s), that we call mode-suppression. We also consider a more general version that suppresses the mode only if the mode frequency is larger than the lowest frequency by a fixed threshold. We prove the stability of mode-suppression using Lyapunov techniques, and use a Kingman bound argument to show that the total download time does not increase with peer arrival rate. We then design versions of mode-suppression that sample a small number of peers at each time, and construct noisy mode estimates by aggregating these samples over time. We show numerically that mode suppression stabilizes and outperforms all other recently proposed chunk sharing algorithms, and via integration into BitTorrent implementation operating over the ns-3 that it ensures stable, low sojourn time operation in a real-world setting.

This report analyzes the performance of distributed Medium Access Control (MAC) protocols in ultra-dense multichannel wireless networks, where$N$frequency bands (or channels) are shared by$M=mN$devices, and devices make decisions to probe and then transmit over available frequency bands. While such a system can be formulated as an$M$-player Bayesian game, it is often infeasible to compute the Nash equilibria of a large-scale system due tothe curse of dimensionality. In this report, we exploit the Mean Field Game (MFG) approach and analyze the system in the large population regime ($N$tends to$\infty $and$m$is a constant). We consider a distributed and low complexity MAC protocol where each device probes$d/k$channels by following an exponential clock which ticks with rate$k$when it has a message to transmit, and optimizes the probing strategy to balance throughput and probing cost. We present a comprehensive analysis from the MFG perspective, including the existence and uniqueness of and convergence to the Mean Field Nash Equilibrium and the price of anarchy with respect to the global optimal solution. Our analysis shows that the price of anarchy is at most one half, but is close to zero when the traffic load or the probing cost is low. Our numerical results confirm our analysis and show that the MFNE is a good approximation of the$M$-player system. Further, this report demonstrates the novelty of MFG analysis, which can be used to study other distributed MAC protocols in ultra-dense wireless networks.

Wireless Internet access has brought legions of heterogeneous applications all sharing the same resources. However, current wireless edge networks that cater to worst or average case performance lack the agility to best serve these diverse sessions. Simultaneously, software reconfigurable infrastructure has become increasingly mainstream to the point that dynamic per packet and per flow decisions are possible at multiple layers of the communications stack. Exploiting such reconfigurability requires the design of a system that can enable a configuration, measure the impact on the application performance (Quality of Experience), and adaptively select a new configuration. Effectively, this feedback loop is a Markov Decision Process whose parameters are unknown. The goal of this work is to design, develop and demonstrate QFlow that instantiates this feedback loop as an application of reinforcement learning (RL). Our context is that of reconfigurable (priority) queueing, and we use the popular application of video streaming as our use case. We develop both model-free and model-based RL approaches that are tailored to the problem of determining which clients should be assigned to which queue at each decision period. Through experimental validation, we show how the RL-based control policies on QFlow are able to schedule the right clients for prioritization in a high-load scenario to outperform the status quo, as well as the best known solutions with over 25% improvement in QoE, and a perfect QoE score of 5 over 85% of the time.

This paper analyzes the performance of distributed Medium Access Control (MAC) protocols in ultra-dense multichannel wireless networks, where N frequency bands (or channels) are shared by M = mN devices, and devices make decisions to probe and then transmit over available frequency bands. While such a system can be formulated as an M-player Bayesian game, it is often infeasible to compute the Nash equilibria of a large-scale system due to the curse of dimensionality. In this paper, we exploit the Mean Field Game (MFG) approach and analyze the system in the large population regime (N tends to ∞ and m is a constant). We consider a distributed and low complexity MAC protocol where each device probes d/k channels by following an exponential clock which ticks with rate k when it has a message to transmit, and optimizes the probing strategy to balance throughput and probing cost. We present a comprehensive analysis from the MFG perspective, including the existence and uniqueness of the Mean Field Nash Equilibrium (MFNE), convergence to the MFNE, and the price of anarchy with respect to the global optimal solution. Our analysis shows that the price of anarchy is at most one half, but is close to zero when the traffic load or the probing cost is low. Our numerical results confirm our analysis and show that the MFNE is a good approximation of the M-player system. Besides showing the efficiency of the considered MAC for emerging applications in ultra-dense multichannel wireless networks, this paper demonstrates the novelty of MFG analysis, which can be used to study other distributed MAC protocols in ultra-dense wireless networks.

We study the problem of broadcasting realtime flows in multi-hop wireless networks. We assume that each packet has a stringent deadline, and each node in the network obtains some utility based on the number of packets delivered to it on time for each flow. We propose a distributed protocol called the delegated-set routing (DSR) that incurs virtually no overhead of coordination among nodes. We also develop distributed algorithms that aim to maximize the total timely utility under DSR. The utility of the DSR protocol and distributed algorithms are demonstrated by both theoretical analysis and simulation results. We show that our algorithms achieve higher timely throughput even when compared against centralized throughput optimal policies that do not consider deadline constraints.

Wireless edge networks are promising to provide better video streaming services to mobile users by provisioning computing and storage resources at the edge of wireless network. However, due to the diversity of user interests, user devices, video versions or resolutions, cache sizes, network conditions, etc., it is challenging to decide where to place the video contents, and which cache and video version a mobile user device should select. In this paper, we study the joint optimization of cache-version selection and content placement for adaptive video streaming in wireless edge networks. We propose practical distributed algorithms that operate at each user device and each network cache to maximize the overall network utility. In addition to proving the optimality of our algorithms, we implement our algorithms as well as several baseline algorithms on ndnSIM, an ns-3 based Named Data Networking simulator. Simulation evaluations demonstrate that our algorithms significantly outperform conventional heuristic solutions.

The ability of a P2P network to scale its throughput up in proportion to the arrival rate of peers has recently been shown to be crucially dependent on the chunk sharing policy employed. Some policies can result in low frequencies of a particular chunk, known as the missing chunk syndrome, which can dramatically reduce throughput and lead to instability of the system. For instance, commonly used policies that nominally “boost” the sharing of infrequent chunks such as the well-known rarest-first algorithm have been shown to be unstable. Recent efforts have largely focused on the careful design of boosting policies to mitigate this issue. We take a complementary viewpoint, and instead consider a policy that simply prevents the sharing of the most frequent chunk(s). Following terminology from statistics wherein the most frequent value in a data set is called the mode, we refer to this policy as mode suppression. We prove the stability of this algorithm using Lyapunov techniques. We also design a distributed version that suppresses the mode via an estimate obtained by sampling three randomly selected peers. We show numerically that both algorithms perform well at minimizing total download times, with distributed mode suppression outperforming all others that we tested against.

We consider a general small-scale market for agent-to-agent resource sharing, in which each agent could either be a server (seller) or a client (buyer) in each time period. In every time period, a server has a certain amount of resources that any client could consume, and randomly gets matched with a client. Our target is to maximize the resource utilization in such an agent-to-agent market, where the agents are strategic. During each transaction, the server gets money and the client gets resources. Hence, trade ratio maximization implies efficiency maximization of our system. We model the proposed market system through a Mean Field Game approach and prove the existence of the Mean Field Equilibrium, which can achieve an almost 100% trade ratio. Finally, we carry out a simulation study motivated by an agent-to-agent computing market, and a case study on a proposed photovoltaic market, and show the designed market benefits both individuals and the system as a whole.

This paper considers the problem of cooperative data exchange with selfish users. In this setting, each user has a subset of packets in the ground set$X$, and wants all other packets in$X$. The users can exchange coded combinations of their packets over a lossless broadcast channel, and monetary transactions are allowed between any pair of users. We define the utility of each user as the sum of two functions: (i) the difference between the total payment received by the user and the total transmission rate of the user, and (ii) the difference between the total number of required packets by the user and the total payment made by the user. A rate-vector and payment-matrix pair$(r,p)$is said to stabilize the grand coalition (i.e., the set of all users) if$(r,p)$is Pareto optimal over all minor coalitions (i.e., all proper subsets of users who collectively know all packets in$X$). Our goal is to design a stabilizing rate-payment pair with minimum sum-rate and minimum sum-payment for any given problem instance. In this work, we show that such a solution always exists, and we propose two algorithms to find such a solution. Moreover, we show that both algorithms maximize the sum utility of all users, while one also maximizes the minimum utility among all users.

An increasing number of applications that will be supported by next generation wireless networks require packets to arrive before a certain deadline for the system to have the desired performance. While many time-sensitive scheduling protocols have been proposed, few have been experimentally evaluated to establish realistic performance. Furthermore, some of these protocols involve high complexity algorithms that need to be performed on a per-packet basis. Experimental evaluation of these protocols requires a flexible platform that is readily capable of implementing and experimenting with these protocols.

Ultra-low per-packet latency has become an essential system requirement as well as a critical challenge for wireless networks. While there is a rich literature on real-time wireless scheduling, it is still unclear what the minimum achievable latency is and what level of throughput can be obtained in practice. This demo presents PULS, a processor-supported software-defined wireless testbed that supports ultra-low-latency scheduling protocols. We will demonstrate that PULS provides strict per-packet latency guarantees as low as 1 millisecond with realistic throughput for wireless networks.

Typical analysis of content caching algorithms using the metric of steady state hit probability under a stationary request process does not account for performance loss under a variable request arrival process. In this paper, we instead conceptualize caching algorithms as complexity-limited online distribution learning algorithms and use this vantage point to study their adaptability from two perspectives: 1) the accuracy of learning a fixed popularity distribution and 2) the speed of learning items' popularity. In order to attain this goal, we compute the distance between the stationary distributions of several popular algorithms with that of a genie-aided algorithm that has the knowledge of the true popularity ranking, which we use as a measure of learning accuracy. We then characterize the mixing time of each algorithm, i.e., the time needed to attain the stationary distribution, which we use as a measure of learning efficiency. We merge both the above-mentioned measures to obtain the “learning error” representing both how quickly and how accurately an algorithm learns the optimal caching distribution and use this to determine the trade-off between these two objectives of many popular caching algorithms. Informed by the results of our analysis, we propose a novel hybrid algorithm, adaptive-least recently used, that learns both faster and better the changes in the popularity. We show numerically that it also outperforms all other candidate algorithms when confronted with either a dynamically changing synthetic request process or using real world traces.

We consider the problem of streaming live content to a cluster of co-located wireless devices that have both an expensive unicast base-station-to-device (B2D) interface, as well as an inexpensive broadcast device-to-device (D2D) interface, which can be used simultaneously. Our setting is a streaming system that uses a block-by-block random linear coding approach to achieve a target percentage of on-time deliveries with minimal B2D usage. Our goal is to design an incentive framework that would promote such cooperation across devices, while ensuring good quality of service. Based on the ideas drawn from truth-telling auctions, we design a mechanism that achieves this goal via appropriate transfers (monetary payments or rebates) in a setting with a large number of devices, and with peer arrivals and departures. Here, we show that a mean field game can be used to accurately approximate our system. Furthermore, the complexity of calculating the best responses under this regime is low. We implement the proposed system on an Android testbed, and illustrate its efficient performance using real world experiments.

We model an Internet marketplace using a set of servers that choose prices for performing jobs. Each server has a queue of unfinished jobs, and is penalized for delay by the market maker via a holding cost. A server completes jobs with a low or high "quality", and jobs truthfully report the quality with which they were completed. The best estimate of quality based on these reports is the "reputation" of the server. A server bases its pricing decision on the distribution of its competitors offered prices and reputations. An entering job is given a random sample of servers, and chooses the best one based on a linear combination of price and reputation. We seek to understand how prices would be determined in such a marketplace using the theory of Mean Field Games. We show the existence of a Mean Field Equilibrium and show how reputation plays a role in allowing servers to declare larger prices than their competitors. We illustrate our results by a numerical study of the system via simulation with parameters chosen from data gathered from existing Internet marketplaces.

With the breakdown of Dennard’s scaling over the past decade, performance growth of modern microprocessor design has largely relied on scaling core count in chip multiprocessors (CMPs). The challenge of chip power density, however, remains and demands new power management solutions. This work investigates a coordinated CMP systemwide Dynamic Voltage and Frequency Scaling (DVFS) policy centered around shared resource utilization. This approach represents a new angle on the problem, differing from the conventional core-workload-driven approaches. The key component of our work is per-core DVFS leveraging a technique similar to TCP Vegas congestion control from networking. This TCP Vegas–based DVFS can potentially identify the synergy between power reduction and performance improvement. Further, this work includes uncore (on-chip interconnect and shared last level cache) and main memory DVFS policies coordinated with the per-core DVFS policy. Full system simulations on PARSEC benchmarks show that our technique reduces total energy dissipation by over 47% across all benchmarks with less than 2.3% performance degradation. Our work also leads to 12% more energy savings compared to a prior work CMP DVFS policy.

We consider the problem of streaming live content to a cluster of co-located wireless devices that have both an expensive unicast base-station-to-device (B2D) interface, as well as an inexpensive broadcast device-to-device (D2D) interface, which can be used simultaneously. Our setting is a streaming system that uses a block-by-block random linear coding approach to achieve a target percentage of on-time deliveries with minimal B2D usage. Our goal is to design an incentive framework that would promote such cooperation across devices, while ensuring good quality of service. Based on ideas drawn from truth-telling auctions, we design a mechanism that achieves this goal via appropriate transfers (monetary payments or rebates) in a setting with a large number of devices, and with peer arrivals and departures. Here, we show that a Mean Field Game can be used to accurately approximate our system. Furthermore, the complexity of calculating the best responses under this regime is low. We implement the proposed system on an Android testbed, and illustrate its efficient performance using real world experiments.

Typically in computer systems, performance must be traded-off to achieve energy savings or, conversely, performance gains come with significant energy overhead. Here, we present a novel approach that can achieve synergistic energy-savings and performance gain in chip multiprocessors (CMPs). Our key observation is that per-core dynamic voltage/frequency scaling (DVFS) can be used as a client regulation mechanism for shared resources on-die. Based on this observation, we propose a new DVFS technique inspired by TCP Vegas, a congestion control protocol from the IP-networking domain. Full system simulations on PARSEC benchmarks show that our technique reduces total CMP energy dissipation by over 40% with a small performance improvement.

No abstract available.

It has been well established that wireless network coding can significantly improve the efficiency of multihop wireless networks. However, in a stochastic environment, some of the packets might not have coding pairs, which limits the number of available coding opportunities. In this context, an important decision is whether to delay packet transmission in hope that a coding pair will be available in the future or transmit a packet without coding. This paper addresses this problem by establishing a stochastic dynamic framework whose objective is to minimize a long-run average cost. We identify an optimal control policy that minimizes the costs due to a combination of transmissions and packet delays. We show that the optimal policy would be stationary, deterministic, and threshold-type based on queue lengths. Our analytical approach is applicable for many cases of interest such as time-varying on/off channels. We further substantiate our results with simulation experiments for more generalized settings.

There has been much work done on designing cellular scheduling algorithms. These algorithms are set up in the manner of a direct mechanism in which the queues reveal their states (backlog), and the scheduler chooses an allocation. In policies such as longest queue first (LQF), scheduling can be shown to yield short queue lengths. However, these algorithms are reliant on truthful declarations of state. Our goal in this article is to determine if a Vickrey–Clarke–Groves (VCG)-type mechanism (a second price auction) that elicits truthful value will also possess the desired LQF-like behavior in a system of dynamically evolving queues. Our approach is to use the concept of a mean field game under which at each time instant, a queue chooses its bid as a best response to its belief that the bids of the others will be drawn independently from a common bid distribution. If this best response is itself a sample from the belief bid distribution, a mean field equilibrium (MFE) is said to exist. We show the existence of the MFE and find it using its structure that the results of the allocation policy are the same as LQF. Thus, the desired LQF-like behavior arises naturally under the second-price auction mechanism. We also present results on the accuracy of the model as the number of agents (queues) becomes large. Finally, using simulations with a large number of queues, we show that computation of the MFE is straightforward.

Networks-on-Chip (NoCs) offer a scalable means of on-chip communication for future many-core chips. This work explores NoC router microarchitectures which leverage traffic pattern biases and imbalances to reduce latency and improve throughput. It introduces STORM, a new, low-latency, fair, highth-roughput NoC router design, customized for the traffic seen in a two-dimensional mesh network employing dimension-order routing. Compared to a baseline NoC router with equivalent buffer resources, STORM offers single cycle operation and reduced cycle time (17% less than the baseline on 45nm CMOS). This design yields a higher overall network saturation throughput (13% higher than the baseline) in an 8x8 2D mesh network for uniform random traffic. STORM also reduces packet latencies under realistic workloads by 36% on average.

We consider a relay node that stochastically receives packets from two opposing flows. Whenever opportunities exist, the relay performs network coding to efficiently transmit packets. However, on one hand, because of the stochastic nature, as well as possible asymmetry between the opposing flows, it would not be possible to always code packets. On the other hand, waiting for a coding opportunity could result in excessive latency, and one may be better off transmitting packets without coding. Thus, one needs to decide at each transmission opportunity whether to transmit a packet uncoded or wait for a future transmission opportunity. To enable us to optimally make that decision, we consider costs for transmission and delay, and formulate our problem as a Markov decision process. We show that the optimal policy is threshold type under a sufficient condition, and we compute it by modeling the resulting system as a Markov chain. Through numerical analysis, we show the effectiveness of the threshold policy in the relay node network, as well as in a line network scenario. Further, we compare the threshold policy against a number of simple heuristic policies and identify situations where these policies can be effective.

The rapid growth of wireless content access implies the need for content placement and scheduling at wireless base stations. We study a system under which users are divided into clusters based on their channel conditions, and their requests are represented by different queues at logical front ends. Requests might be elastic (implying no hard delay constraint) or inelastic (requiring that a delay target be met). Correspondingly, we have request queues that indicate the number of elastic requests, and deficit queues that indicate the deficit in inelastic service. Caches are of finite size and can be refreshed periodically from a media vault. We consider two cost models that correspond to inelastic requests for streaming stored content and real-time streaming of events, respectively. We design provably optimal policies that stabilize the request queues (hence ensuring finite delays) and reduce average deficit to zero [hence ensuring that the quality-of-service (QoS) target is met] at small cost. We illustrate our approach through simulations.

A large number of congestion control protocols have been proposed in the last few years, with all having the same purpose—to divide available bandwidth resources among different flows in a fair manner. Each protocol operates on the paradigm of some conception of link price (such as packet losses or packet delays) that determines source transmission rates. Recent work on network utility maximization has brought forth the idea that the fundamental price or Lagrange multiplier for a link is proportional to the queue length at that link, and that different congestion metrics (such as delays or drops) are essentially ways of interpreting such a Lagrange multiplier. We thus ask the following question: Suppose that each flow has a number of congestion control protocols to choose from, which one (or combination) should it choose? We introduce a framework wherein each flow has a utility that depends on throughput and also has a disutility that is some function of the queue lengths encountered along the route taken. Flows must choose a combination of protocols that would maximize their payoffs. We study both the socially optimal, as well as the selfish cases to determine the loss of system-wide value incurred through selfish decision making, so characterizing the “price of heterogeneity.” We also propose tolling schemes that incentivize flows to choose one of several different virtual networks catering to particular needs and show that the total system value is greater, hence making a case for the adoption of such virtual networks.

We consider wireless networks in which multiple paths are available between each source and destination. We allow each source to split traffic among all of its available paths, and we ask the question: How do we attain the lowest possible number of transmissions per unit time to support a given traffic matrix? Traffic bound in opposite directions over two wireless hops can utilize the “reverse carpooling” advantage of network coding in order to decrease the number of transmissions used. We call such coded hops “hyper-links.” With the reverse carpooling technique, longer paths might be cheaper than shorter ones. However, there is a peculiar situation among sources—the network coding advantage is realized only if there is traffic in both directions of a shared path. We consider the problem of routing with network coding by selfish agents (the sources) as a potential game and develop a method of state-space augmentation in which additional agents (the hyper-links) decouple sources' choices from each other by declaring a hyper-link capacity, allowing sources to split their traffic selfishly in a distributed fashion, and then changing the hyper-link capacity based on user actions. Furthermore, each hyper-link has a scheduling constraint in terms of the maximum number of transmissions allowed per unit time. We show that our two-level control scheme is stable and verify our analytical insights by simulation.

We consider a group of co-located wireless peer devices that desire to synchronously receive a live content stream. The devices are each equipped with an expensive unicast base-station-to-device (B2D) interface, as well as a broadcast device-to-device (D2D) interface over a shared medium. The stream is divided into blocks, which must be played out soon after their initial creation. If a block is not received within a specific time after its creation, it is rendered useless and dropped. The blocks in turn are divided into random linear coded chunks to facilitate sharing across the devices. We transform the problem into the two questions of (i) deciding which peer should broadcast a chunk on the D2D channel at each time, and (ii) how long B2D transmissions should take place for each block. We analytically develop a provably-minimum-cost algorithm that can ensure that QoS targets can be met for each device. We study its performance via simulations, and present an overview of our implementation on Android phones using the algorithm as a basis.

The rapid increase of content delivery over the Internet has led to the proliferation of content distribution networks (CDNs). Management of CDNs requires algorithms for request routing, content placement, and eviction in such a way that user delays are small. We abstract the system of frontend source nodes and backend caches of the CDN in the likeness of the input and output nodes of a switch. In this model, queues of requests for different pieces of content build up at the source nodes, which route these requests to a cache that contains the requested content. For each request that is routed to a cache, a corresponding data file is transmitted back to the requesting source across links of finite capacity. Caches are of finite size, and the content of the caches can be refreshed periodically. Our objective is to design policies for request routing, content placement and content eviction with the goal of small user delays. Stable policies ensure the finiteness of the request queues, while good polices also lead to short queue lengths. We first design a throughput-optimal algorithm that solves the routing-placement-eviction problem. The design yields insight into the impact of different cache refresh policies on queue length, and we construct throughput optimal algorithms that engender short queue lengths. We illustrate the potential of our approach through simulations on different CDN topologies.

A large number of congestion control protocols have been proposed in the last few years, with all having the same purpose-to divide available bandwidth resources among different flows in a fair manner. Each protocol operates on the paradigm of some conception of link price (such as packet losses or packet delays) that determines source transmission rates. Recent work on network utility maximization has brought forth idea that the fundamental price or Lagrange multiplier for a link is proportional the queue length at that link, and that different congestion metrics (such as delays or drops) are essentially ways of interpreting such a Lagrange multiplier. We thus ask the following question: Suppose that each flow has a number of congestion control protocols to choose from, which one (or combination) should it choose? We introduce a framework wherein each flow has a utility that depends on throughput, and also has a disutility that is some function of the queue lengths encountered along the route taken. Flows must choose a combination of protocols that would maximize their payoffs. We study both the socially optimal, as well as the selfish cases to determine the loss of system-wide value incurred through selfish decision making, so characterizing the “price of heterogeneity”. We also propose tolling schemes that incentivize flows to choose one of several different virtual networks catering to particular needs, and show that the total system value is greater, hence making a case for the adoption of such virtual networks.

It has been well established that reverse-carpooling based network coding can significantly improve the efficiency of multi-hop wireless networks. However, in a stochastic environment when there are no opportunities to code because of packets without coding pairs, should these packets wait for a future opportunity or should they be transmitted without coding? To help answer that question we formulate a stochastic dynamic program with the objective of minimizing the long-run average cost per unit time incurred due to transmissions and delays. In particular, we develop optimal control actions that would balance between costs of transmission against those of delays. In that process we seek to address a crucial question: what should be observed as the state of the system? We analytically show that just the queue lengths is enough if it can be modeled as a Markov process. Subsequently we show that a stationary policy based on queue lengths is optimal and describe a procedure to find such a policy. We further substantiate our results with simulation experiments for more generalized settings.

The rapid growth of wireless content access implies the need for content placement and scheduling at wireless base stations. We study a system under which clients are divided into clusters based on their channel conditions, and their requests are represented by different queues at logical frontends. Requests might be elastic (implying no hard delay constraint) or inelastic (requiring that a delay target be met). Correspondingly, we have request queues that indicate the number of elastic requests, and deficit queues that indicate the deficit in inelastic service. Caches are of finite size, and can be refreshed periodically from a media vault. We design provably optimal policies that stabilize the request queues (hence ensuring finite delays) and reduce average deficit to zero (hence ensuring that the QoS target is met). We illustrate our approach through simulations.

The traditional formulation of the total value of information transfer is a multi-commodity flow problem. Each data source is seen as generating a commodity along a fixed route, and the objective is to maximize the total system throughput under some concept of fairness, subject to capacity constraints of the links used. This problem is well studied under the framework of network utility maximization and has led to several different distributed congestion control schemes. However, this view of value does not capture the fact that flows may associate value, not just with throughput, but with link-quality metrics such as packet delay and jitter. In this work, the congestion control problem is redefined to include individual source preferences. It is assumed that degradation in link quality seen by a flow adds up on the links it traverses, and the total utility is maximized in such a way that the end-to-end quality degradation seen by each source is bounded by a value that it declares. Decoupling source-dissatisfaction and link-degradation through an effective capacity variable, a distributed and provably optimal resource allocation algorithm is designed to maximize system utility subject to these quality constraints. The applicability of the controller in different situations is supported by numerical simulations, and a protocol developed using the controller is simulated on ns-2 to illustrate its performance.

We take an analytical approach to study fundamental rate-delay-reliability trade-offs in the context of media streaming. We consider the probability of interruption in media playback (buffer underflow) as well as the number of initially buffered packets (initial waiting time) as the Quality of user Experience (QoE) metrics. We characterize the optimal trade-off between these metrics as a function of system parameters such as the packet arrival rate and file size, for different channel models. In the first model, we assume packets arrive according to independent Poisson processes from multiple servers or peers. We use random linear network coding to simplify the packet requests at the network layer and avoid duplicate packet reception. This allows us to model the receiver's buffer as a queue with Poisson arrivals and deterministic departures. For this model, we show that for arrival rates slightly larger than the play rate, the minimum initial buffering required to achieve certain level of interruption probability remains bounded as the file size grows. This is not the case when the arrival rate and the play rate match. In the second model, we consider channels with memory, which can be modeled using Markovian arrival processes. We characterize the optimal trade-off curves for the infinite file size case, in such Markovian environments.

The growth of real-time content streaming over the Internet has resulted in the use of peer-to-peer (P2P) approaches for scalable content delivery. In such P2P streaming systems, each peer maintains a playout buffer of content chunks which it attempts to fill by contacting other peers in the network. The objective is to ensure that the chunk to be played out is available with high probability while keeping the buffer size small. A small playout buffer means that the playout delay is small. Thus, the objective is to study the tradeoff between two measures of QoS, chunk playout rate and delay. A policy is a rule that suggests which chunks should be requested by the peer from other peers. We consider consider a number of recently suggested policies consistent with buffer minimization for a given target of skip-free playout. We first study a rarest-first policy that attempts to obtain chunks farthest from playout, and a greedy policy that attempts to obtain chunks nearest to playout. We show that they both have similar buffer scalings (as a function of the number of peers of target probability of skip-free probability). We then study a hybrid policy which achieves order sense improvements over both policies and can achieve order optimal performance. We validate our results using simulations.

The traditional formulation of the total value of information transfer is a multi-commodity flow problem. Here, each data source is seen as generating a commodity along a fixed route, and the objective is to maximize the total system throughput under some concept of fairness, subject to capacity constraints of the links used. This problem is well studied under the framework of network utility maximization and has led to several different distributed congestion control schemes. However, this idea of value does not capture the fact that flows might associate value, not just with throughput, but with link-quality metrics such as packet delay, jitter and so on. The traditional congestion control problem is redefined to include individual source preferences. It is assumed that degradation in link quality seen by a flow adds up on the links it traverses, and the total utility is maximized in such a way that the quality degradation seen by each source is bounded by a value that it declares. Decoupling source-dissatisfaction and link- degradation through an ``effective capacity'' variable, a distributed and provably optimal resource allocation algorithm is designed, to maximize system utility subject to these quality constraints. The applicability of our controller in different situations is illustrated, and results are supported through numerical examples.

We consider wireless networks in which multiple paths are available between each source and destination. We allow each source to split traffic among all of its available paths, and ask the question: how do we attain the lowest possible number of transmissions per unit time to support a given traffic matrix? Traffic bound in opposite directions over two wireless hops can utilize the ``reverse carpooling'' advantage of network coding in order to decrease the number of transmissions used. We call such coded hops as ``hyper-links''. With the reverse carpooling technique longer paths might be cheaper than shorter ones. However, there is a prisoners dilemma type situation among sources -- the network coding advantage is realized only if there is traffic in both directions of a shared path. We develop a two-level distributed control scheme that decouples user choices from each other by declaring a hyper- link capacity, allowing sources to split their traffic selfishly in a distributed fashion, and then changing the hyper-link capacity based on user actions. We show that such a controller is stable, and verify our analytical insights by simulation.

We take an analytical approach to study Quality of user Experience (QoE) for media streaming applications. We use the fact that random linear network coding applied to blocks of video frames can significantly simplify the packet requests at the network layer and avoid duplicate packet reception. We model the receiver's buffer as a queue with Poisson arrivals and deterministic departures. We consider the probability of interruption in video playback (buffer underflow) as well as the number of initially buffered packets (initial waiting time) as the QoE metrics. We explicitly characterize the optimal trade-off between these metrics by providing upper and lower bounds on the minimum initial buffering required to achieve certain level of interruption probability for different regimes of the system parameters. Our bounds are asymptotically tight as the file size goes to infinity. Further, we show that for arrival rates slightly larger than the play rate, the minimum initial buffering remains bounded as the file size grows. This is not the case when the arrival rate and the play rate match.

We consider wireless ad hoc networks with a large number of users. Subsets of users might be interested in identical information, and so we have a regime in which several multicast sessions may coexist. We first calculate an upper bound on the achievable transmission rate per multicast flow as a function of the number of multicast sources in such a network. We then propose a simple comb-based architecture for multicast routing, which achieves the upper bound in an order sense under certain constraints. Compared to the approach of constructing a Steiner tree to decide multicast paths, our construction achieves the same order-optimal results while requiring little location information and no computational overhead.

We study revenue-maximizing pricing by a service provider in a communication network and compare revenues from simple pricing rules to the maximum revenues that are feasible. In particular, we focus on flat entry fees as the simplest pricing rule. We provide a lower bound for the ratio between the revenue from this pricing rule and maximum revenue, which we refer to as the Price of Simplicity. We characterize what types of environments lead to a low Price of Simplicity and show that in a range of environments, the loss of revenue from using simple entry fees is small. We then study the Price of Simplicity for a simple non-linear pricing (price discrimination) scheme based on the Paris Metro Pricing. The service provider creates different service classes and charges differential entry fees for these classes. We show that the gain from this type of price discrimination is small, particularly in environments in which the simple entry fee pricing leads to a low Price of Simplicity.

We consider wireless ad hoc networks with a large number of users. Subsets of users might be interested in identical information, and so we have a regime in which several multicast sessions may coexist. We first calculate an upper-bound on the achievable transmission rate per multicast flow as a function of the number of multicast sources in such a network. We then propose a simple comb-based architecture for multicast routing which achieves the upper bound in an order sense under certain constraints. Compared to the approach of constructing a Steiner tree to decide multicast paths, our construction achieves the same order-optimal results while requiring little location information and no computational overhead.

We consider non-cooperative mobiles, each faced with the problem of which subset of WLANs access points (APs) to connect and multihome to, and how to split its traffic among them. Considering the many users regime, we obtain a potential game model and study its equilibrium. We obtain pricing for which the total throughput is maximized at equilibrium and study the convergence to equilibrium under various evolutionary dynamics. We also study the case where the Internet service provider (ISP) could charge prices greater than that of the cost price mechanism and show that even in this case multihoming is desirable.

Internet worms, which spread in computer networks without human mediation, pose a severe threat to computer systems today. The rate of propagation of worms has been measured to be extremely high and they can infect a large fraction of their potential hosts in a short time. We study two different methods of patch dissemination to combat the spread of worms. We first show that using a fixed number of patch servers performs inadequately against Internet worms. We then show that by exploiting the exponential data dissemination capability of P2P systems, the spread of worms can be halted effectively. We compare the two methods by using fluid models to compute two quantities of interest: the time taken to effectively combat the progress of the worm, and the maximum number of infected hosts. We validate our models using simulations.

Abstract — In many cases, a mobile user has the option of connecting to one of several IEEE 802.11 access points (APs), each using an independent channel. User throughput in each AP is determined by the number of other users as well as the frame size and physical rate being used. We consider the scenario where users could multihome, i.e., split their traffic amongst all the available APs, based on the throughput they obtain and the price charged. Thus, they are involved in a non-cooperative game against each other. We convert the problem into a fluid model and show that under a pricing scheme, which we call the cost price mechanism, the total system throughput is maximized, i.e., the system suffers no loss of efficiency due to selfish dynamics. We also study the case where the Internet Service Provider (ISP) could charge prices greater than that of the cost price mechanism. We show that even in this case multihoming outperforms unihoming, both in terms of throughput as well as profit to the ISP. I.

In this paper we examine how transit and customer prices are set in a network consisting of multiple ISPs. Some ISPs may be geographically co-located so that they compete for the same set of end users. We examine the existence of equilibrium price strategies in this situation and show how positive profit can be achieved using threat strategies. It is shown that if the number of ISPs competing for the same customers is large then it can lead to price wars. ISPs that are not geographically co-located may not directly compete for users, but are nevertheless involved in a non-cooperative game of setting access and transit prices for each other. We study how such ISPs are linked economically through transit ISPs by considering a multi-stage game. We also consider the economics of private exchange points and show that they could become far more wide spread then they currently are.