This paper studies a variation of the classic network alignment problem, named diffusion-network alignment. The goal is to align the vertices of a rooted diffusion tree to the vertices of a network, where the diffusion tree could be from a communication trace or contact tracing, and the network could be an online or offline social network. Different from the classic network alignment where both networks are fully observed, this model captures the information asymmetry of two networks. To solve this problem, this paper presents an efficient algorithm based on tree correlation tests to extract alignment information from local neighborhoods. We analyze the performance of the algorithm in the sparse graph regime and show that with high probability, all matched pairs are correct. Furthermore, for each vertex on the diffusion tree, this paper establishes an explicit lower bound on the probability that the vertex is correctly matched. These lower bounds are depth-dependent and increase as vertices get closer to the root.

This paper studies a class of multi-agent reinforcement learning (MARL) problems where the reward that an agent receives depends on the states of other agents, but the next state only depends on the agent’s own current state and action. We name it REC-MARL standing for REward-Coupled Multi-Agent Reinforcement Learning. REC-MARL has a range of important applications such as real-time access control and distributed power control in wireless networks. This paper presents a distributed policy gradient algorithm for REC-MARL. The proposed algorithm isdistributedin two aspects: (i) the learned policy is a distributed policy that maps a local state of an agent to its local action and (ii) the learning/training is distributed, during which each agent updates its policy based on its own and neighbors’ information. The learned algorithm achievesa stationary policyand its iterative complexity bounds depend on the dimension of local states and actions. The experimental results of our algorithm for the real-time access control and power control in wireless networks show that our policy significantly outperforms the state-of-the-art algorithms and well-known benchmarks.

Reward inference (learning a reward model from human preferences) is a critical intermediate step in the Reinforcement Learning from Human Feedback (RLHF) pipeline for fine-tuning Large Language Models (LLMs). In practice, RLHF faces fundamental challenges such as distribution shift, reward model overfitting, and problem misspecification. An alternative approach is direct policy optimization without reward inference, such as Direct Preference Optimization (DPO), which provides a much simpler pipeline and has shown empirical success in LLM applications. However, DPO utilizes the closed-form expression between the optimal policy and the reward function, which is only suitable under the bandit setting or deterministic MDPs. This paper develops two RLHF algorithms without reward inference for general RL problems beyond bandits and deterministic MDPs, and general preference models beyond the Bradley-Terry model. The key idea is to estimate the local value function difference from human preferences and then approximate the policy gradient with a zeroth-order gradient approximator. For both algorithms, we establish polynomial convergence rates in terms of the number of policy gradient iterations, the number of trajectory samples, and human preference queries per iteration. Numerical experiments in stochastic environments validate the performance of our proposed algorithms, outperforming popular RLHF baselines such as DPO and PPO. Our paper shows there exist provably efficient methods to solve general RLHF problems without reward inference.

We study a two-sided market, wherein, price-sensitive heterogeneous customers and servers arrive and join their respective queues. A compatible customer-server pair can then be matched by the platform, at which point, they leave the system. Our objective is to design pricing and matching algorithms that maximize the platform's profit, while maintaining reasonable queue lengths. As the demand and supply curves governing the price-dependent arrival rates may not be known in practice, we design a novel online-learning-based pricing policy and establish its near-optimality. In particular, we prove a tradeoff among three performance metrics: $\tilde{O}(T^{1-\gamma})$ regret, $\tilde{O}(T^{\gamma/2})$ average queue length, and $\tilde{O}(T^{\gamma})$ maximum queue length for $\gamma \in (0, 1/6]$, significantly improving over existing results (Yang & Ying, 2024). Moreover, barring the permissible range of $\gamma$, we show that this trade-off between regret and average queue length is optimal up to logarithmic factors under a class of policies, matching the optimal one as in (Varma et al., 2023) which assumes the demand and supply curves to be known. Our proposed policy has two noteworthy features: a dynamic component that optimizes the tradeoff between low regret and small queue lengths; and a probabilistic component that resolves the tension between obtaining useful samples for fast learning and maintaining small queue lengths.

Network alignment, which aims to find node correspondence across different networks, is the cornerstone of various downstream multi-network and Web mining tasks. Most of the embedding-based methods indirectly model cross-network node relationships by contrasting positive and negative node pairs sampled from hand-crafted strategies, which are vulnerable to graph noises and lead to potential misalignment of nodes. Another line of work based on the optimal transport (OT) theory directly models cross-network node relationships and generates noise-reduced alignments. However, OT methods heavily rely on fixed, pre-defined cost functions that prohibit end-to-end training and are hard to generalize. In this paper, we aim to unify the embedding and OT-based methods in a mutually beneficial manner and propose a joint optimal transport and embedding framework for network alignment named JOENA. For one thing (OT for embedding), through a simple yet effective transformation, the noise-reduced OT mapping serves as an adaptive sampling strategy directly modeling all cross-network node pairs for robust embedding learning. For another (embedding for OT), on top of the learned embeddings, the OT cost can be gradually trained in an end-to-end fashion, which further enhances the alignment quality. With a unified objective, the mutual benefits of both methods can be achieved by an alternating optimization schema with guaranteed convergence. Extensive experiments on real-world networks validate the effectiveness and scalability of JOENA, achieving up to 16% improvement in MRR and 20 times speedup compared with the state-of-the-art alignment methods.

Lung cancer remains the leading cause of cancer-related death worldwide, and early diagnosis of lung cancer is critical for improving the survival rate of patients. Performing annual low-dose computed tomography (LDCT) screening among high-risk populations is the primary approach for early diagnosis. However, after each screening, whether to continue monitoring (with follow-up screenings) or to order a biopsy for diagnosis remains a challenging decision to make. Continuing with follow-up screenings may lead to delayed diagnosis but ordering a biopsy without sufficient evidence incurs unnecessary risk and cost. In this paper, we tackle the problem by an optimal stopping approach. Our proposed algorithm, called EarlyStop-RL, utilizes the structure of the Snell envelope for optimal stopping, and model-free deep reinforcement learning for making diagnosis decisions. Through evaluating our algorithm on a commonly used clinical trial dataset (the National Lung Screening Trial), we demonstrate that EarlyStop-RL has the potential to greatly enhance risk assessment and early diagnosis of lung cancer, surpassing the performance of two widely adopted clinical models, namely the Lung-RADS and the Brock model.

This paper studies safe Reinforcement Learning (safe RL) with linear function approximation and under hard instantaneous constraints where unsafe actions must be avoided at each step. Existing studies have considered safe RL with hard instantaneous constraints, but their approaches rely on several key assumptions: (i) the RL agent knows a safe action set for every state or knows a safe graph in which all the state-action-state triples are safe, and (ii) the constraint/cost functions are linear. In this paper, we consider safe RL with instantaneous hard constraints without assumption (i) and generalize (ii) to Reproducing Kernel Hilbert Space (RKHS). Our proposed algorithm, LSVI-AE, achieves O(√{d³H⁴K}) regret and O(H √{dK}) hard constraint violation when the cost function is linear and O(H?ₖ √{K}) hard constraint violation when the cost function belongs to RKHS. Here K is the learning horizon, H is the length of each episode, and ?ₖ is the information gain w.r.t the kernel used to approximate cost functions. Our results achieve the optimal dependency on the learning horizon K, matching the lower bound we provide in this paper and demonstrating the efficiency of LSVI-AE. Notably, the design of our approach encourages aggressive policy exploration, providing a unique perspective on safe RL with general cost functions and no prior knowledge of safe actions, which may be of independent interest.

Mixup, which generates synthetic training samples on the data manifold, has been shown to be highly effective in augmenting Euclidean data. However, finding a proper data manifold for graph data is non-trivial, as graphs are non-Euclidean data in disparate spaces. Though efforts have been made, most of the existing graph mixup methods neglect the intrinsic geodesic guarantee, thereby generating inconsistent sample-label pairs. To address this issue, we propose GeoMix to mixup graphs on the Gromov-Wasserstein (GW) geodesics. A joint space over input graphs is first defined based on the GW distance, and graphs are then transformed into the GW space through equivalence-preserving transformations. We further show that the linear interpolation of the transformed graph pairs defines a geodesic connecting the original pairs on the GW manifold, hence ensuring the consistency between generated samples and labels. An accelerated mixup algorithm on the approximate low-dimensional GW manifold is further proposed. Extensive experiments show that the proposed GeoMix promotes the generalization and robustness of GNN models.

This paper proposes new joint flow control and link scheduling (JFCLS) algorithms for the classical network utility maximization (NUM) problem with unknown utility functions. Our algorithm leverages the idea of optimism, i.e., being optimistic in using the historical information to predict the future impact of flow rate and link scheduling decisions, to reduce the oscillation in the flow rate and link scheduling decision. The optimistic design leads to a gradient-type update for flow rate control. We prove that optimistic JFCLS with the gradient information of utility functions establishes a zero optimal utility gap with O(1/T) convergence rate while guaranteeing a constant queue length at each node for any time slot. When only the values of utility functions are observed, we propose zero-order optimistic JFCLS and prove it establishes a trade-off with utility gap O(1/T¼) and O(T¾) queue length. Our experiments demonstrate the proposed optimistic algorithm achieves a fast convergence rate and is very adaptive to network dynamics, such as flow dynamics or link failure.

The traditional multi-armed bandit (MAB) model for recommendation systems assumes the user stays in the system for the entire learning horizon. In new online education platforms such as ALEKS or new video recommendation systems such as TikTok, the amount of time a user spends on the app depends on how engaging the recommended contents are. Users may temporarily leave the system if the recommended items cannot engage the users. To understand the exploration, exploitation, and engagement in these systems, we propose a new model, called MAB-A where “A” stands for abandonment and the abandonment probability depends on the current recommended item and the user's past experience (called state). We propose two algorithms, ULCB and KL-ULCB, both of which do more exploration (being optimistic) when the user likes the previous recommended item and less exploration (being pessimistic) when the user does not. We prove that both ULCB and KL-ULCB achieve logarithmic regret, $O(\log K)$, where $K$ is the number of visits (or episodes). Furthermore, the regret bound under KL-ULCB is asymptotically sharp. We also extend the proposed algorithms to the general-state setting. Simulation results show that the proposed algorithms have significantly lower regret than the traditional UCB and KL-UCB, and Q-learning-based algorithms.

Existing approaches for vehicle repositioning on large-scale ride-hailing platforms either ignore the spatial-temporal mismatch between supply and demand in real-time or overlook the long-term balance of the system. To account for both, we propose a lookahead repositioning policy in this paper, which is a novel approach to repositioning idle vehicles from both a dynamic system and a long-term performance perspective. Our method consists of two parts; the first part utilizes linear programming (LP) to formulate the nonstationary system as a time-varying,$T$-step lookahead optimization problem and explicitly models the fraction of drivers who follow repositioning recommendations (called the repositioning rate). The second step is to incorporate a reinforcement learning (RL) method to maximize long-term return based on learned value functions after the$T$time slots. Extensive studies utilizing a real-world dataset on both small-scale and large-scale simulators show that our method outperforms previous baseline methods and is robust to prediction errors.

We study model-free reinforcement learning (RL) algorithms in episodic non-stationary constrained Markov decision processes (CMDPs), in which an agent aims to maximize the expected cumulative reward subject to a cumulative constraint on the expected utility (cost). In the non-stationary environment, the reward, utility functions, and the transition kernels can vary arbitrarily over time as long as the cumulative variations do not exceed certain variation budgets. We propose the first model-free, simulator-free RL algorithms with sublinear regret and zero constraint violation for non-stationary CMDPs in both tabular and linear function approximation settings with provable performance guarantees. Our results on regret bound and constraint violation for the tabular case match the corresponding best results for stationary CMDPs when the total budget is known. Additionally, we present a general framework for addressing with the well-known challenges associated with analyzing non-stationary CMDPs, without requiring prior knowledge of the variation budget. We apply the approach for both tabular and linear approximation settings.

Multi-server queueing systems are widely used models for job scheduling in machine learning, wireless networks, and crowdsourcing. This paper considers a multi-server system with multiple servers and multiple types of jobs, where different job types require different amounts of processing time at different servers. The goal is to schedule jobs on servers without knowing the statistics of the processing times. To fully utilize the processing power of the servers, it is known that one has to at least learn the service rates of different job types on different servers. Prior works on this topic decouple the learning and scheduling phases which leads to either excessive exploration or extremely large job delays. We propose a new algorithm, which combines the MaxWeight scheduling policy with discounted upper confidence bound (UCB), to simultaneously learn the statistics and schedule jobs to servers. We obtain performance bounds for our algorithm that hold for both stationary and nonstationary service rates. Simulations confirm that the delay performance of our algorithm is several orders of magnitude better than previously proposed algorithms. Our algorithm also has the added benefit that it can handle non-stationarity in the service processes.

This paper studies online nonstochastic control problems with adversarial and static constraints. We propose online nonstochastic control algorithms that achieve both sublinear regret and sublinear adversarial constraint violation while keeping static constraint violation minimal against the optimal constrained linear control policy in hindsight. To establish the results, we introduce an online convex optimization with memory framework under adversarial and static constraints, which serves as a subroutine for the constrained online nonstochastic control algorithms. This subroutine also achieves the state-of-the-art regret and constraint violation bounds for constrained online convex optimization problems, which is of independent interest. Our experiments demonstrate the proposed control algorithms are adaptive to adversarial constraints and achieve smaller cumulative costs and violations. Moreover, our algorithms are less conservative and achieve significantly smaller cumulative costs than the state-of-the-art algorithm.

Risk-sensitive reinforcement learning (RL) has become a popular tool to control the risk of uncertain outcomes and ensure reliable performance in various sequential decision-making problems. While policy gradient methods have been developed for risk-sensitive RL, it remains unclear if these methods enjoy the same global convergence guarantees as in the risk-neutral case. In this paper, we consider a class of dynamic time-consistent risk measures, called Expected Conditional Risk Measures (ECRMs), and derive policy gradient updates for ECRM-based objective functions. Under both constrained direct parameterization and unconstrained softmax parameterization, we provide global convergence and iteration complexities of the corresponding risk-averse policy gradient algorithms. We further test risk-averse variants of REINFORCE and actor-critic algorithms to demonstrate the efficacy of our method and the importance of risk control.

Diffusion on graphs is ubiquitous with numerous high-impact applications, ranging from the study of residential segregation in socioeconomics and activation cascading in neuroscience, to the modeling of disease contagion in epidemiology and malware spreading in cybersecurity. In these applications, complete diffusion histories play an essential role in terms of identifying dynamical patterns, reflecting on precaution actions, and forecasting intervention effects. Despite their importance, complete diffusion histories are rarely available and are highly challenging to reconstruct due to ill-posedness, explosive search space, and scarcity of training data. To date, few methods exist for diffusion history reconstruction. They are exclusively based on the maximum likelihood estimation (MLE) formulation and require to know true diffusion parameters. In this paper, we study an even harder problem, namely reconstructing Diffusion history from A single SnapsHot (DASH), where we seek to reconstruct the history from only the final snapshot without knowing true diffusion parameters. We start with theoretical analyses that reveal a fundamental limitation of the MLE formulation. We prove: (a) estimation error of diffusion parameters is unavoidable due to NP-hardness of diffusion parameter estimation, and (b) the MLE formulation is sensitive to estimation error of diffusion parameters. To overcome the inherent limitation of the MLE formulation, we propose a novel barycenter formulation: finding the barycenter of the posterior distribution of histories, which is provably stable against the estimation error of diffusion parameters. We further develop an effective solver named DIffusion hiTting Times with Optimal proposal (DITTO) by reducing the problem to estimating posterior expected hitting times via the Metropolis-Hastings Markov chain Monte Carlo method (M-H MCMC) and employing an unsupervised graph neural network to learn an optimal proposal to accelerate the convergence of M-H MCMC. We conduct extensive experiments to demonstrate the efficacy of the proposed method. Our code is available at https://github.com/q-rz/KDD23-DITTO. The appendix can be found at https://arxiv.org/abs/2306.00488.

Fair resource allocation is one of the most important topics in communication networks. Existing solutions almost exclusively assume each user utility function is known and concave. This paper seeks to answer the following question: how to allocate resources when utility functions are unknown, even to the users? This answer has become increasingly important in the next-generation AI-aware communication networks where the user utilities are complex and their closed-forms are hard to obtain. In this paper, we provide a new solution using a distributed and data-driven bilevel optimization approach, where the lower level is a distributed network utility maximization (NUM) algorithm with concave surrogate utility functions, and the upper level is a data-driven learning algorithm to find the best surrogate utility functions that maximize the sum of true network utility. The proposed algorithm learns from data samples (utility values or gradient values) to autotune the surrogate utility functions to maximize the true network utility, so works for unknown utility functions. For the general network, we establish the nonasymptotic convergence rate of the proposed algorithm with nonconcave utility functions. The simulations validate our theoretical results and demonstrate the great effectiveness of the proposed method in a real-world network.

This paper considers a class of reinforcement learning problems, which involve systems with two types of states: stochastic and pseudo-stochastic. In such systems, stochastic states follow a stochastic transition kernel while the transitions of pseudo-stochastic states are deterministic {\em given} the stochastic states/transitions. We refer to such systems as mixed systems, which are widely used in various applications, including Manufacturing systems, communication networks, and queueing networks. We propose a sample-efficient RL method that accelerates learning by generating augmented data samples. The proposed algorithm is data-driven (model-free), but it learns the policy from data samples from both real and augmented samples. This method significantly improves learning by reducing the sample complexity such that the dataset only needs to have sufficient coverage of the stochastic states. We analyze the sample complexity of the proposed method under Fitted Q Iteration (FQI) and demonstrate that the optimality gap decreases as $O\left(\sqrt{\frac{1}{n}}+\sqrt{\frac{1}{m}}\right),$ where $n$ represents the number of real samples, and $m$ is the number of augmented samples per real sample. It is important to note that without augmented samples, the optimality gap is $O(1)$ due to the insufficient data coverage of the pseudo-stochastic states. Our experimental results on multiple queueing network applications confirm that the proposed method indeed significantly accelerates both deep Q-learning and deep policy gradient.

This paper considers a stochastic Multi-Armed Bandit (MAB) problem with dual objectives: (i) quick identification and commitment to the optimal arm, and (ii) reward maximization throughout a sequence of $T$ consecutive rounds. Though each objective has been individually well-studied, i.e., best arm identification for (i) and regret minimization for (ii), the simultaneous realization of both objectives remains an open problem, despite its practical importance. This paper introduces \emph{Regret Optimal Best Arm Identification} (ROBAI) which aims to achieve these dual objectives. To solve ROBAI with both pre-determined stopping time and adaptive stopping time requirements, we present an algorithm called EOCP and its variants respectively, which not only achieve asymptotic optimal regret in both Gaussian and general bandits, but also commit to the optimal arm in $\mathcal{O}(\log T)$ rounds with pre-determined stopping time and $\mathcal{O}(\log^2 T)$ rounds with adaptive stopping time. We further characterize lower bounds on the commitment time (equivalent to the sample complexity) of ROBAI, showing that EOCP and its variants are sample optimal with pre-determined stopping time, and almost sample optimal with adaptive stopping time. Numerical results confirm our theoretical analysis and reveal an interesting ``over-exploration'' phenomenon carried by classic UCB algorithms, such that EOCP has smaller regret even though it stops exploration much earlier than UCB, i.e., $\mathcal{O}(\log T)$ versus $\mathcal{O}(T)$, which suggests over-exploration is unnecessary and potentially harmful to system performance.

Multi-sourced networks naturally appear in many application domains, ranging from bioinformatics, social networks, neuroscience to management. Although state-of-the-art offers rich models and algorithms to find various patterns when input networks are given, it has largely remained nascent on how vulnerable the mining results are due to the adversarial attacks. In this paper, we address the problem of attacking multi-network mining through the way of deliberately perturbing the networks to alter the mining results. The key idea of the proposed method admiring is effective and efficient influence functions on the Sylvester equation defined over the input networks, which plays a central and unifying role in various multi-network mining tasks. The proposed algorithms bear three main advantages, including (1) effectiveness, being able to accurately quantify the rate of change of the mining results in response to attacks; (2) efficiency, scaling linearly with more than 100 times speed-up over the straight-forward implementation without any quality loss; and (3) generality, being applicable to a variety of multi-network mining tasks (e.g., graph kernel, network alignment, cross-network node similarity) with different attacking strategies (e.g., edge/node removal, attribute alteration).

We consider an ultra-dense wireless network with$N$channels and$M = N$devices. Messages with fresh information are generated at each device according to a random process and need to be transmitted to an access point. The value of a message decreases as it ages, so each device searches for an idle channel to transmit the message as soon as it can. However, each channel probing is associated with a fixed cost (energy), so a device needs to adapt its probing rate based on the “age” of the message. At each device, the design of the optimal probing strategy can be formulated as an infinite horizon Markov Decision Process (MDP) where the devices compete with each other to find idle channels. While it is natural to view the system as a Bayesian game, it is often intractable to analyze such a system. Thus, we use the Mean Field Game (MFG) approach to analyze the system in a large-system regime, where the number of devices is very large, to understand the structure of the problem and to find efficient probing strategies. We present an analysis based on the MFG perspective. We begin by characterizing the space of valid policies and use this to show the existence of a Mean Field Nash Equilibrium (MFNE) in a constrained set for any general increasing cost functions with diminishing rewards. Further we provide an algorithm for computing the equilibrium for any given device, and the corresponding age-dependent channel probing policy.

Mobile cloud computing enables compute-limited mobile devices to perform real-time intensive computations such as speech recognition or object detection by leveraging powerful cloud servers. An important problem in large-scale mobile cloud computing is computational offloading, where each mobile device decides when and how much computation should be uploaded to cloud servers by considering the local processing delay and the cost of using cloud servers. In this paper, we develop a distributed threshold-based offloading algorithm where it uploads an incoming computing task to cloud servers if the number of tasks queued at the device reaches the threshold and processes it locally otherwise. The threshold is updated iteratively based on the computational load and the cost of using cloud servers. We formulate the problem as a symmetric game, and characterize the sufficient and necessary conditions for the existence and uniqueness of the Nash Equilibrium (NE) assuming exponential service times. Then, we show the convergence of our proposed distributed algorithm to the NE when the NE exists. Further, we characterize the performance gap between cost under our proposed distributed algorithm and the minimum cost in terms of Price of Anarchy (PoA) when the cost of using cloud servers is high. Finally, we perform extensive simulations to validate our theoretical findings, demonstrate the efficiency of our proposed distributed algorithm under various scenarios such as hyperexponential service times, imperfect server utilization estimation, and asynchronous threshold updates, and reveal the superior performance of threshold-based policies over their probabilistic counterpart.

This paper presents a model-free reinforcement learning (RL) algorithm for infinite-horizon average-reward Constrained Markov Decision Processes (CMDPs). Considering a learning horizon K, which is sufficiently large, the proposed algorithm achieves sublinear regret and zero constraint violation. The bounds depend on the number of states S, the number of actions A, and two constants which are independent of the learning horizon K.

Graph neural networks (GNNs) have achieved tremendous success in many graph learning tasks such as node classification, graph classification and link prediction. For the classification task, GNNs' performance often highly depends on the number of labeled nodes and thus could be significantly hampered due to the expensive annotation cost. The sparse literature on active learning for GNNs has primarily focused on selecting only one sample each iteration, which becomes inefficient for large scale datasets. In this paper, we study the batch active learning setting for GNNs where the learning agent can acquire labels of multiple samples at each time. We formulate batch active learning as a cooperative multi-agent reinforcement learning problem and present a novel reinforced batch-mode active learning framework BiGeNe. To avoid the combinatorial explosion of the joint action space, we introduce a value decomposition method that factorizes the total Q-value into the average of individual Q-values. Moreover, we propose a novel multi-agent Q-network consisting of a graph convolutional network (GCN) component and a gated recurrent unit (GRU) component. The GCN component takes both the informativeness and inter-dependences between nodes into account and the GRU component enables the agent to consider interactions between selected nodes in the same batch. Experimental results on multiple public datasets demonstrate the effectiveness and efficiency of our proposed method.

This paper presents the first model-free, simulator-free reinforcement learning algorithm for Constrained Markov Decision Processes (CMDPs) with sublinear regret and zero constraint violation. The algorithm is named Triple-Q because it includes three key components: a Q-function (also called action-value function) for the cumulative reward, a Q-function for the cumulative utility for the constraint, and a virtual-Queue that (over)-estimates the cumulative constraint violation. Under Triple-Q, at each step, an action is chosen based on the pseudo-Q-value that is a combination of the three “Q” values. The algorithm updates the reward and utility Q-values with learning rates that depend on the visit counts to the corresponding (state, action) pairs and are periodically reset. In the episodic CMDP setting, Triple-Q achieves $\tilde{\cal O}\left(\frac{1 }{\delta}H^4 S^{\frac{1}{2}}A^{\frac{1}{2}}K^{\frac{4}{5}} \right)$ regret, where $K$ is the total number of episodes, $H$ is the number of steps in each episode, $S$ is the number of states, $A$ is the number of actions, and $\delta$ is Slater’s constant. Furthermore, {Triple-Q} guarantees zero constraint violation, both on expectation and with a high probability, when $K$ is sufficiently large. Finally, the computational complexity of {Triple-Q} is similar to SARSA for unconstrained MDPs, and is computationally efficient.

Recent years have witnessed the superior performance of heterogeneous graph neural networks (HGNNs) in dealing with heterogeneous information networks (HINs). Nonetheless, the success of HGNNs often depends on the availability of sufficient labeled training data, which can be very expensive to obtain in real scenarios. Active learning provides an effective solution to tackle the data scarcity challenge. For the vast majority of the existing work regarding active learning on graphs, they mainly focus on homogeneous graphs, and thus fall in short or even become inapplicable on HINs. In this paper, we study the active learning problem with HGNNs and propose a novel meta-reinforced active learning framework MetRA. Previous reinforced active learning algorithms train the policy network on labeled source graphs and directly transfer the policy to the target graph without any adaptation. To better exploit the information from the target graph in the adaptation phase, we propose a novel policy transfer algorithm based on meta-Q-learning termed per-step MQL. Empirical evaluations on HINs demonstrate the effectiveness of our proposed framework. The improvement over the best baseline is up to 7% in Micro-F1.

We consider the problem of controlling a mutated diffusion process with an unknown mutation time. The problem is formulated as the quickest intervention problem with the mutation modeled by a change-point, which is a generalization of the quickest change-point detection (QCD). Our goal is to intervene in the mutated process as soon as possible while maintaining a low intervention cost with optimally chosen intervention actions. This model and the proposed algorithms can be applied to pandemic prevention (such as Covid-19) or misinformation containment. We formulate the problem as a partially observed Markov decision process (POMDP) and convert it to an MDP through the belief state of the change-point. We first propose a grid-approximation approach to calculate the optimal intervention policy, whose computational complexity could be very high when the number of grids is large. In order to reduce the computational complexity, we further propose a low-complexity threshold-based policy through the analysis of the first-order approximation of the value functions in the "local intervention" regime. Simulation results show the low-complexity algorithm has a similar performance as the grid-approximation approach and both perform much better than the QCD-based algorithms.

This paper considers online convex optimization with hard constraints and analyzes achievable regret and cumulative hard constraint violation (violation for short). The problem distinguishes itself from online convex optimization with soft constraints, where a violation at one round can be compensated/cancelled by a conservative decision at a different round. We propose a RECtified Online Optimization algorithm (RECOO) and consider two settings: fixed constraints and adversarial constraints. Both settings have been considered in the literature. Compared with existing results, {\em RECOO achieves the best of two worlds and beyond.} For the fixed-constraints setting, RECOO achieves $O\left(\sqrt{T}\right)$ regret and $O(1)$ violation, where $T$ is the learning horizon. The best known results in this case are $O(\sqrt{T})$ regret and $O\left(T^{1/4}\right)$ violation. For the adversarial-constraints setting, it guarantees $O(\sqrt{T})$ regret and $O(T^{3/4})$ violation, which match the best existing results. When the loss functions are strongly convex, RECOO can guarantee $O(\log T)$ regret and $O(1)$ violation for fixed constraints, and $O(\log T)$ regret and $O(\sqrt{T\log T})$ violation for adversarial constraints. Both these results are order-wise better than the existing bounds. The regret and violation bounds mentioned above use the best fixed decision in hindsight as the baseline. This paper further considers a dynamic baseline where the comparator sequence is time-varying. This paper shows that RECOO not only improves the existing results in the fixed-constraints setting but also {\em for the first time,} guarantees dynamic regret and violation bounds in the adversarial-constraints setting. Our experiment results confirm that RECOO outperforms several existing algorithms for both fixed and adversarial constraints.

Bilevel optimization has arisen as a powerful tool for solving a variety of machine learning problems. Two current popular bilevel optimizers AID-BiO and ITD-BiO naturally involve solving one or two sub-problems, and consequently, whether we solve these problems with loops (that take many iterations) or without loops (that take only a few iterations) can significantly affect the overall computational efficiency. Existing studies in the literature cover only some of those implementation choices, and the complexity bounds available are not refined enough to enable rigorous comparison among different implementations. In this paper, we first establish unified convergence analysis for both AID-BiO and ITD-BiO that are applicable to all implementation choices of loops. We then specialize our results to characterize the computational complexity for all implementations, which enable an explicit comparison among them. Our result indicates that for AID-BiO, the loop for estimating the optimal point of the inner function is beneficial for overall efficiency, although it causes higher complexity for each update step, and the loop for approximating the outer-level Hessian-inverse-vector product reduces the gradient complexity. For ITD-BiO, the two loops always coexist, and our convergence upper and lower bounds show that such loops are necessary to guarantee a vanishing convergence error, whereas the no-loop scheme suffers from an unavoidable non-vanishing convergence error. Our numerical experiments further corroborate our theoretical results.

Collision-free path planning is a major challenge in managing unmanned aerial vehicles (UAVs) fleets, especially in uncertain environments. In this paper, we consider the design of UAV routing policies using multi-agent reinforcement learning, and propose a Multi-resolution, Multi-agent, Mean-field reinforcement learning algorithm, named3M-RL,for flight planning, where multiple vehicles need to avoid collisions with each other while moving towards their destinations. In the system we consider, each UAV makes decisions based on local observations, and does not communicate with other UAVs. The algorithm trains a routing policy using an Actor-Critic neural network with multi-resolution observations, including detailed local information and aggregated global information based on mean-field. The algorithm tackles the curse-of-dimensionality problem in multi-agent reinforcement learning and provides a scalable solution. We test our algorithm in different complex scenarios in both 2D and 3D space and our simulation results show that 3M-RL result in good routing policies.

This paper considers the steady-state performance of load balancing algorithms in a many-server system with distributed queues. The system has$N$servers, and each server maintains a local queue with buffer size$b-1$, i.e. a server can hold at most one job in service and$b-1$jobs in the queue. Jobs in the same queue are served according to the first-in-first-out (FIFO) order. The system is operated in a heavy-traffic regime such that the workload per server is$\lambda = 1 - N^{-\alpha }$for$0.5\leq \alpha < 1$. We identify a set of algorithms such that the steady-state queues have the following universal scaling, whereuniversalmeans that it holds for any$\alpha \in [0.5,1$): (i) the number of busy servers is$\lambda N-o(1)$; and (ii) the number of servers with two jobs (one in service and one in queue) is$O(N^{\alpha }\log N)$; and (iii) the number of servers with more than two jobs is$O({1}/{N^{r(1-\alpha)-1}})$, where$r$can be any positive integer independent of$N$. The set of load balancing algorithms that satisfy the sufficient condition includes join-the-shortest-queue (JSQ), idle-one-first (I1F), and power-of-$d$-choices (Po$d$) with$d\geq 2N^\alpha \log N$. We further argue that the waiting time of such an algorithm is near optimal order-wise.

We consider an ultra-dense wireless network with N channels and M = N devices. Messages with fresh information are generated at each device according to a random process and need to be transmitted to an access point. The value of a message decreases as it ages, so each device searches for an idle channel to transmit the message as soon as it can. However, each channel probing is associated with a fixed cost (energy), so a device needs to adapt its probing rate based on the "age" of the message. At each device, the design of the optimal probing strategy can be formulated as an infinite horizon Markov Decision Process (MDP) where the devices compete with each other to find idle channels. While it is natural to view the system as a Bayesian game, it is often intractable to analyze such a system. Thus, we use the Mean Field Game (MFG) approach to analyze the system in a large-system regime, where the number of devices is very large, to understand the structure of the problem and to find efficient probing strategies. We present an analysis based on the MFG perspective. We begin by characterizing the space of valid policies and use this to show the existence of a Mean Field Nash Equilibrium (MFNE) in a constrained set for any general increasing cost functions with diminishing rewards. Further we provide an algorithm for computing the equilibrium for any given device, and the corresponding age-dependent channel probing policy.

Mobile cloud computing enables compute-limited mobile devices to perform real-time intensive computations such as speech recognition or object detection by leveraging powerful cloud servers. An important problem in large-scale mobile cloud computing is computational offloading where each mobile device decides when and how much computation should be uploaded to cloud servers by considering the local processing delay and the cost of using cloud servers. In this paper, we develop a distributed threshold-based offloading algorithm where it uploads an incoming computing task to cloud servers if the number of tasks queued at the device reaches the threshold, and processes it locally otherwise. The threshold is updated iteratively based on the computational load and the cost of using cloud servers. We formulate the problem as a symmetric game, and characterize the sufficient and necessary conditions for the existence and uniqueness of the Nash Equilibrium (NE) assuming exponential service times. Then, we show the convergence of our proposed distributed algorithm to the NE when the NE exists. Finally, we perform extensive simulations to validate our theoretical findings and demonstrate the efficiency of our proposed distributed algorithm under various practical scenarios such as general service times, imperfect server utilization estimation, and asynchronous threshold updates.

This paper provides a recipe for deriving calculable approximation errors of mean-field models in heavy-traffic with the focus on the well-known load balancing algorithm --- power-of-two-choices (Po2). The recipe combines Stein's method for linearized mean-field models and State Space Concentration (SSC) based on geometric tail bounds. In particular, our approach divides the state space into two regions, a neighborhood near the mean-field equilibrium and the complement of that. We first use a tail bound to show that the steady-state probability being outside the neighborhood is small. Then, we use a linearized mean-field model and Stein's method to characterize the generator difference, which provides the dominant term of the approximation error. From the dominant term, we are able to obtain an asymptotically-tight bound and a nonasymptotic upper bound, both are calculable bounds, not order-wise scaling results like most results in the literature. Finally, we compare the theoretical bounds with numerical evaluations to show the effectiveness of our results. We note that the simulation results show that both bounds are valid even for small size systems such as a system with only ten servers.

This paper considers stochastic linear bandits with general nonlinear constraints. The objective is to maximize the expected cumulative reward over horizon $T$ subject to a set of constraints in each round $\tau\leq T$. We propose a pessimistic-optimistic algorithm for this problem, which is efficient in two aspects. First, the algorithm yields $\tilde{\cal O}\left(\left(\frac{K^{0.75}}{\delta}+d\right)\sqrt{\tau}\right)$ (pseudo) regret in round $\tau\leq T,$ where $K$ is the number of constraints, $d$ is the dimension of the reward feature space, and $\delta$ is a Slater's constant; and {\em zero} constraint violation in any round $\tau>\tau',$ where $\tau'$ is {\em independent} of horizon $T.$ Second, the algorithm is computationally efficient. Our algorithm is based on the primal-dual approach in optimization and includes two components. The primal component is similar to unconstrained stochastic linear bandits (our algorithm uses the linear upper confidence bound algorithm (LinUCB)). The computational complexity of the dual component depends on the number of constraints, but is independent of the sizes of the contextual space, the action space, and the feature space. Thus, the computational complexity of our algorithm is similar to LinUCB for unconstrained stochastic linear bandits.

Network alignment finds node correspondences across multiple networks, where the alignment accuracy is of crucial importance because of its profound impact on downstream applications. The vast majority of existing works focus on how to best utilize the topology and attribute information of the input networks as well as the anchor links when available. Nonetheless, it has not been well studied on how to boost the alignment performance through actively obtaining high-quality and informative anchor links, with a few exceptions. The sparse literature on active network alignment introduces the human in the loop to label some seed node correspondence (i.e., anchor links), which are informative from the perspective of querying the most uncertain node given few potential matchings. However, the direct influence of the intrinsic network attribute information on the alignment results has largely remained unknown. In this paper, we tackle this challenge and propose an active network alignment method (Attent) to identify the best nodes to query. The key idea of the proposed method is to leverage effective and efficient influence functions defined over the alignment solution to evaluate the goodness of the candidate nodes for query. Our proposed query strategy bears three distinct advantages, including (1) effectiveness, being able to accurately quantify the influence of the candidate nodes on the alignment results; (2) efficiency, scaling linearly with 15 − 17 × speed-up over the straight-forward implementation without any quality loss; (3) generality, consistently improving alignment performance of a variety of network alignment algorithms.

The connectivity of networks has been widely studied in many high-impact applications, ranging from immunization, critical infrastructure analysis, social network mining, to bioinformatic system studies. Regardless of the end application domains, connectivity minimization has always been a fundamental task to effectively control the functioning of the underlying system. The combinatorial nature of the connectivity minimization problem imposes an exponential computational complexity to find the optimal solution, which is intractable in large systems. To tackle the computational barrier, greedy algorithm is extensively used to ensure a near-optimal solution by exploiting the diminishing returns property of the problem. Despite the empirical success, the theoretical and algorithmic challenges of the problems still remain wide open. On the theoretical side, the intrinsic hardness and the approximability of the general connectivity minimization problem are still unknown except for a few special cases. On the algorithmic side, existing algorithms are hard to balance between the optimization quality and computational efficiency. In this article, we address the two challenges by (1) proving that the general connectivity minimization problem is NP-hard and is the best approximation ratio for any polynomial algorithms, and (2) proposing the algorithm CONTAIN and its variant CONTAIN + that can well balance optimization effectiveness and computational efficiency for eigen-function based connectivity minimization problems in large networks.

In this paper, we establish a theoretical comparison between the asymptotic mean square errors of double Q-learning and Q-learning. Our result builds upon an analysis for linear stochastic approximation based on Lyapunov equations and applies to both tabular setting or with linear function approximation, provided that the optimal policy is unique and the algorithms converge. We show that the asymptotic mean-square error of Double Q-learning is exactly equal to that of Q-learning if Double Q-learning uses twice the learning rate of Q-learning and the output of Double Q-learning is the average of its two estimators. We also present some practical implications of this theoretical observation using simulations.

Understanding diffusion processes in networks has emerged as an important research topic because of its wide range of applications. Analysis of diffusion traces can help us answer important questions such as the source(s) of diffusion and the role of each node during the diffusion process. However, in large-scale networks, due to the cost and privacy concerns, it is almost impossible to monitor the entire network and collect the complete diffusion trace. In this paper, we tackle the problem of reconstructing the diffusion history from a partial observation. We formulate the diffusion history reconstruction problem as a maximum a posteriori (MAP) problem and prove the problem is NP-hard. Then, we propose a step-by-step reconstruction algorithm, which can always produce a diffusion history that is consistent with the partial observation. Our experimental results based on synthetic and real networks show that the algorithm significantly outperforms some existing methods.

We study Waze-inspired spectrum discovery, where the cloud collects the spectrum sensing results from many smartphones and predicts location-specific spectrum availability based on information fusion. Observe that with limited sensing capability, each smartphone can sense only a limited number of channels; and further, the more channels each smartphone senses, the less accurate the sensing results would be. In particular, we consider two different smartphone sensing models: a homogeneous model and a heterogeneous model. To develop a comprehensive understanding, we cast the spectrum discovery problem as a matrix recovery problem, which is different from the classical matrix completion problem, in the sense that it suffices to determine only part of the matrix entries in the matrix recovery formulation. It is shown that the widely-used similarity-based collaborative filtering method would not work well because it requires each smartphone to sense too many channels. With this motivation, we propose a location-aided smartphone data fusion method and show that the channel numbers each smartphone needs to sense could be dramatically reduced. Moreover, we analyze the partial matrix recovery performance by using the location-aided data fusion method. Both theoretical analysis and numerical results corroborate the intuition that with each smartphone sensing more channels, the recovery performance improves at first but then degrades beyond some point because of the decreasing sensing accuracy.

This report analyzes the performance of distributed Medium Access Control (MAC) protocols in ultra-dense multichannel wireless networks, where$N$frequency bands (or channels) are shared by$M=mN$devices, and devices make decisions to probe and then transmit over available frequency bands. While such a system can be formulated as an$M$-player Bayesian game, it is often infeasible to compute the Nash equilibria of a large-scale system due tothe curse of dimensionality. In this report, we exploit the Mean Field Game (MFG) approach and analyze the system in the large population regime ($N$tends to$\infty $and$m$is a constant). We consider a distributed and low complexity MAC protocol where each device probes$d/k$channels by following an exponential clock which ticks with rate$k$when it has a message to transmit, and optimizes the probing strategy to balance throughput and probing cost. We present a comprehensive analysis from the MFG perspective, including the existence and uniqueness of and convergence to the Mean Field Nash Equilibrium and the price of anarchy with respect to the global optimal solution. Our analysis shows that the price of anarchy is at most one half, but is close to zero when the traffic load or the probing cost is low. Our numerical results confirm our analysis and show that the MFNE is a good approximation of the$M$-player system. Further, this report demonstrates the novelty of MFG analysis, which can be used to study other distributed MAC protocols in ultra-dense wireless networks.

This paper presents a novel algorithm for recovering missing values of co-evolving time series with partial embedded network information. The idea is to connect two sources of data (time series data and embedded network data) through a shared low dimensional latent space. The proposed algorithm, named NetDyna, is an Expectation-Maximization (EM) algorithm, and uses the Kalman filter and matrix factorization approaches to infer the missing values both in the time series and embedded network. Our experimental results on real datasets, including a Motes dataset and a Motion Capture dataset, show that (1) NetDyna outperforms other state-of-the-art algorithms, especially with partially observed network information; (2) its computational complexity scales linearly with the time duration of time series; and (3) the algorithm recovers the embedded network in addition to missing time series values.

We consider the dynamics of a linear stochastic approximation algorithm driven by Markovian noise, and derive finite-time bounds on the moments of the error, i.e., deviation of the output of the algorithm from the equilibrium point of an associated ordinary differential equation (ODE). We obtain finite-time bounds on the mean-square error in the case of constant step-size algorithms by considering the drift of an appropriately chosen Lyapunov function. The Lyapunov function can be interpreted either in terms of Stein’s method to obtain bounds on steady-state performance or in terms of Lyapunov stability theory for linear ODEs. We also provide a comprehensive treatment of the moments of the square of the 2-norm of the approximation error. Our analysis yields the following results: (i) for a given step-size, we show that the lower-order moments can be made small as a function of the step-size and can be upper-bounded by the moments of a Gaussian random variable; (ii) we show that the higher-order moments beyond a threshold may be infinite in steady-state; and (iii) we characterize the number of samples needed for the finite-time bounds to be of the same order as the steady-state bounds. As a by-product of our analysis, we also solve the open problem of obtaining finite-time bounds for the performance of temporal difference learning algorithms with linear function approximation and a constant step-size, without requiring a projection step or an i.i.d. noise assumption.

Multi-sourced networks naturally appear in many application domains, ranging from bioinformatics, social networks, neuroscience to management. Although state-of-the-art offers rich models and algorithms to find various patterns when input networks are given, it has largely remained nascent on how vulnerable the mining results are due to the adversarial attacks. In this paper, we address the problem of attacking multi-network mining through the way of deliberately perturbing the networks to alter the mining results. The key idea of the proposed method (Admiring) is effective influence functions on the Sylvester equation defined over the input networks, which plays a central and unifying role in various multi-network mining tasks. The proposed algorithms bear two main advantages, including (1) effectiveness, being able to accurately quantify the rate of change of the mining results in response to attacks; and (2) generality, being applicable to a variety of multi-network mining tasks ( e.g., graph kernel, network alignment, cross-network node similarity) with different attacking strategies (e.g., edge/node removal, attribute alteration).

This paper analyzes the performance of distributed Medium Access Control (MAC) protocols in ultra-dense multichannel wireless networks, where N frequency bands (or channels) are shared by M = mN devices, and devices make decisions to probe and then transmit over available frequency bands. While such a system can be formulated as an M-player Bayesian game, it is often infeasible to compute the Nash equilibria of a large-scale system due to the curse of dimensionality. In this paper, we exploit the Mean Field Game (MFG) approach and analyze the system in the large population regime (N tends to ∞ and m is a constant). We consider a distributed and low complexity MAC protocol where each device probes d/k channels by following an exponential clock which ticks with rate k when it has a message to transmit, and optimizes the probing strategy to balance throughput and probing cost. We present a comprehensive analysis from the MFG perspective, including the existence and uniqueness of the Mean Field Nash Equilibrium (MFNE), convergence to the MFNE, and the price of anarchy with respect to the global optimal solution. Our analysis shows that the price of anarchy is at most one half, but is close to zero when the traffic load or the probing cost is low. Our numerical results confirm our analysis and show that the MFNE is a good approximation of the M-player system. Besides showing the efficiency of the considered MAC for emerging applications in ultra-dense multichannel wireless networks, this paper demonstrates the novelty of MFG analysis, which can be used to study other distributed MAC protocols in ultra-dense wireless networks.

We study two time-scale linear stochastic approximation algorithms, which can be used to model well-known reinforcement learning algorithms such as GTD, GTD2, and TDC. We present finite-time performance bounds for the case where the learning rate is fixed. The key idea in obtaining these bounds is to use a Lyapunov function motivated by singular perturbation theory for linear differential equations. We use the bound to design an adaptive learning rate scheme which significantly improves the convergence rate over the known optimal polynomial decay rule in our experiments, and can be used to potentially improve the performance of any other schedule where the learning rate is changed at pre-determined time instants.

Modern cellular base stations can transmit over multiple frequencies, and further choose to transmit to different users over different frequencies. In much of the prior literature, it is assumed that the channel state of each user over each frequency is known. However, to get such channel state information for each user-channel pair requires a large overhead. Here, we consider the problem of computationally efficient and throughput-optimal scheduling in networks where the base station ensures a small probing overhead by limiting the number of allowed probe packets per time slot. We first argue that a naive optimization-based MaxWeight algorithm is combinatorially infeasible to implement, and then design a low-complexity algorithm that achieves the same throughput as the naive MaxWeight algorithm. Through simulations, we also investigate further improvements to achieve very small packet delays.

The unprecedented amounts of data have catalyzed the trend of combining human insights with machine learning techniques, which facilitate the use of crowdsourcing to enlist label information both effectively and efficiently. One crucial challenge in crowdsourcing is the diverse worker quality, which determines the accuracy of the label information provided by such workers. Motivated by the observations that same set of tasks are typically labeled by the same set of workers, we studied their behaviors across multiple related tasks and proposed an optimization framework for learning from task and worker dual heterogeneity. The proposed method uses a weight tensor to represent the workers’ behaviors across multiple tasks, and seeks to find the optimal solution of the tensor by exploiting its structured information. Then, we propose an iterative algorithm to solve the optimization problem and analyze its computational complexity. To infer the true label of an example, we construct a worker ensemble based on the estimated tensor, whose decisions will be weighted using a set of entropy weight. We also prove that the gradient of the most time-consuming updating block is separable with respect to the workers, which leads to a randomized algorithm with faster speed. Moreover, we extend the learning framework to accommodate to the multi-class setting. Finally, we test the performance of our framework on several datasets, and demonstrate its superiority over state-of-the-art techniques.

This paper studies the problem of robustifying an interdependent network by rewiring a small number of links in realtime during a cascading attack. Interdependent networks have been widely used to model interconnected complex systems such as a critical infrastructure network including both the power grid and the Internet. Realtime robustification of interdependent networks, therefore, has significant practical importance. This paper formulates the problem using the Markov decision process (MDP) framework. We first show the problem is NP-hard and then develop an effective and efficient greedy algorithm, named REALWIRE, to robustify the network in realtime. REALWIREscores each link (and each node) based on the expected number of links failures resulted from the failure of the link (or the node), and rewires the links greedily according to the scores. Extensive experimental results show that REALWIREoutperforms other algorithms on multiple trobustness metrics.

Power-of-d-choices is a popular load balancing algorithm for many-server systems such as large-scale data centers. For each incoming job, the algorithm probes d servers, chosen uniformly at random from a total of N servers, and routes the job to the least loaded one. It is well known that power-of-d-choices reduces queueing delays by orders of magnitude compared to the policy that routes each incoming job to a randomly selected server. The question to be addressed in this paper is how large d needs to be so that power-of-d-choices achieves asymptotic zero delay like the join-the-shortest-queue (JSQ) algorithm, which is a special case of power-of-d-choices with d=N. We are interested in the heavy-traffic regime where the load of the system, denoted by λ, approaches to one as N increases, and assume λ = 1-γN-αfor and . This paper establishes that when d=ω-([1/(1-λ)]), the probability that an incoming job is routed to a busy server is asymptotically zero, i.e. a job experiences zero queueing delay with probability one asymptotically; and when d=O([1/(1-λ)])' the probability that a job is routed to a busy server is lower bounded by a positive constant independent of N. Therefore, our results show that d=ω([1/(1-λ)]) is sufficient and almost necessary for achieving zero delay with the power-of-d-choices load balancing policy.

Network connectivity optimization, which aims to manipulate network connectivity by changing its underlying topology, is a fundamental task behind a wealth of high-impact data mining applications, ranging from immunization, critical infrastructure construction, social collaboration mining, bioinformatics analysis, to intelligent transportation system design. To tackle its exponential computation complexity, greedy algorithms have been extensively used for network connectivity optimization by exploiting its diminishing returns property. Despite the empirical success, two key challenges largely remain open. First, on the theoretic side, the hardness, as well as the approximability of the general network connectivity optimization problem are still nascent except for a few special instances. Second, on the algorithmic side, current algorithms are often hard to balance between the optimization quality and the computational efficiency. In this paper, we systematically address these two challenges for the network connectivity optimization problem. First, we reveal some fundamental limits by proving that, for a wide range of network connectivity optimization problems, (1) they are NP-hard and (2) (1-1/e) is the optimal approximation ratio for any polynomial algorithms. Second, we propose an effective, scalable and general algorithm (CONTAIN) to carefully balance the optimization quality and the computational efficiency.

We study Waze-inspired spectrum discovery, where the cloud collects the spectrum sensing results from many smartphones and predicts location-specific spectrum availability based on information fusion. Observe that with limited sensing capability, each smartphone can sense only a limited number of channels; and further, the more channels each smartphone senses, the less accurate the sensing results would be. To develop a comprehensive understanding, we cast the spectrum discovery problem as a matrix recovery problem, which is different from the classical matrix completion problem, in the sense that it suffices to determine only part of the matrix entries in the matrix recovery formulation. It is shown that the widely-used similarity-based collaborative filtering method would not work well because it requires each smartphone to sense too many channels. With this motivation, we propose a location-aided smartphone data fusion method and show that the channel numbers each smartphone needs to sense could be dramatically reduced. Moreover, we analyze the partial matrix recovery performance by using the location-aided data fusion method, and numerical results corroborate the intuition that with each smartphone sensing more channels, the recovery performance improves at first but then degrades beyond some point because of the decreasing sensing accuracy.

This paper studies the problem of locating multiple diffusion sources in networks with partial observations. We propose a new source localization algorithm, named Optimal-Jordan-Cover (OJC). The algorithm first extracts a subgraph using a candidate selection algorithm that selects source candidates based on the number of observed infected nodes in their neighborhoods. Then, in the extracted subgraph, OJC finds a set of nodes that "cover" all observed infected nodes with the minimum radius. The set of nodes is called the Jordan cover, and is regarded as the set of diffusion sources. Considering the heterogeneous susceptible-infected-recovered (SIR) diffusion in the Erdos-Renyi (ER) random graph, we prove that OJC can locate all sources with probability one asymptotically with partial observations. OJC is a polynomial-time algorithm in terms of network size. However, the computational complexity increases exponentially in m; the number of sources. We further propose a low-complexity heuristic based on the K-Means for approximating the Jordan cover, named Approximate-Jordan-Cover (AJC). Simulations on random graphs and real networks demonstrate that both AJC and OJC significantly outperform other heuristic algorithms.

Nowadays, crowdsourcing has been commonly used to enlist label information both effectively and efficiently. One major challenge in crowdsourcing is the diverse worker quality, which determines the accuracy of the label information provided by such workers. Motivated by the observation that in many crowdsourcing platforms, the same set of workers typically work on the same set of tasks, we propose to model the diverse worker quality by studying their behaviors across multiple related tasks. To this end, we propose an optimization framework named MultiC2 for learning from task and worker dual heterogeneity. It uses a weight tensor to represent the workers' behaviors across multiple tasks, and seeks to find the optimal solution of the tensor by exploiting its structured information. We then propose an iterative algorithm to solve the optimization framework and analyze its computational complexity. To infer the true label of an example, we construct a worker ensemble based on the estimated tensor, whose decisions will be weighted using a set of entropy weight. Finally, we test the performance of MultiC2 on various data sets, and demonstrate its superiority over state-of-the-art crowdsourcing techniques.

The increasingly connected world has catalyzed the fusion of networks from different domains, which facilitates the emergence of a new network model-multi-layered networks. Examples of such kind of network systems include critical infrastructure networks, biological systems, organization-level collaborations, cross-platform e-commerce, and so forth. One crucial structure that distances multi-layered network from other network models is its cross-layer dependency, which describes the associations between the nodes from different layers. Needless to say, the cross-layer dependency in the network plays an essential role in many data mining applications like system robustness analysis and complex network control. However, it remains a daunting task to know the exact dependency relationships due to noise, limited accessibility, and so forth. In this article, we tackle the cross-layer dependency inference problem by modeling it as a collective collaborative filtering problem. Based on this idea, we propose an effective algorithm Fascinate that can reveal unobserved dependencies with linear complexity. Moreover, we derive Fascinate-ZERO, an online variant of Fascinate that can respond to a newly added node timely by checking its neighborhood dependencies. We perform extensive evaluations on real datasets to substantiate the superiority of our proposed approaches.

This paper studies information source detection in networks under the independent cascade (IC) model. Assume the spread of information starts from a single source in a network and a complete snapshot of the network is obtained at some time. The goal is to identify the source based on the observation. We derive the maximum a posterior (MAP) estimator of the source for tree networks and propose a Short-Fat Tree (SFT) algorithm for general networks based on the MAP estimator. The algorithm selects the Jordan infection center [1] and breaks ties according the degree of boundary infected nodes. Loosely speaking, the algorithm selects the node such that the breadth-first search (BFS) tree from it has the minimum depth but the maximum number of leaf nodes. On the Erdos-Renyi (ER) random graph, we establish the following possibility and impossibility results: (i) when the infection duration0.5, SFT identifies the source with probability 1 (w.p.1) asymptotically (as network size increases to infinity), where n is the network size and μ is the average node degree; (ii) when the infection duration > ⌈log n/ log μ ⌉ + 2, the probability of identifying the source approaches zero asymptotically under any algorithm; and (iii) when infection duration0, asymptotically, at least 1-δ fraction of the nodes on the BFS tree starting from the source are leaf-nodes, where δ = 3√log n/μ, i.e., the BFS tree starting from the actual source is a fat tree. 1Numerical experiments on tree networks, the ER random graphs and real world networks with different evaluation metrics show that the SFT algorithm outperforms existing algorithms.

Multi-layered networks have recently emerged as a new network model, which naturally finds itself in many high-impact application domains, ranging from critical inter-dependent infrastructure networks, biological systems, organization-level collaborations, to cross-platform e-commerce, etc. Cross-layer dependency, which describes the dependencies or the associations between nodes across different layers/networks, often plays a central role in many data mining tasks on such multi-layered networks. Yet, it remains a daunting task to accurately know the cross-layer dependency a prior. In this paper, we address the problem of inferring the missing cross-layer dependencies on multi-layered networks. The key idea behind our method is to view it as a collective collaborative filtering problem. By formulating the problem into a regularized optimization model, we propose an effective algorithm to find the local optima with linear complexity. Furthermore, we derive an online algorithm to accommodate newly arrived nodes, whose complexity is just linear wrt the size of the neighborhood of the new node. We perform extensive empirical evaluations to demonstrate the effectiveness and the efficiency of the proposed methods.

We study the value of data privacy in a game-theoretic model of trading private data, where a data collector purchases private data from strategic data subjects (individuals) through an incentive mechanism. The private data of each individual represents her knowledge about an underlying state, which is the information that the data collector desires to learn. Different from most of the existing work on privacy-aware surveys, our model does not assume the data collector to be trustworthy. Then, an individual takes full control of its own data privacy and reports only a privacy-preserving version of her data. In this paper, the value of ε units of privacy is measured by the minimum payment of all nonnegative payment mechanisms, under which an individual's best response at a Nash equilibrium is to report the data with a privacy level of ε. The higher ε is, the less private the reported data is. We derive lower and upper bounds on the value of privacy which are asymptotically tight as the number of data subjects becomes large. Specifically, the lower bound assures that it is impossible to use less amount of payment to buy ε units of privacy, and the upper bound is given by an achievable payment mechanism that we designed. Based on these fundamental limits, we further derive lower and upper bounds on the minimum total payment for the data collector to achieve a given learning accuracy target, and show that the total payment of the designed mechanism is at most one individual's payment away from the minimum.

Mean-field models have been used to study large-scale and complex stochastic systems, such as large-scale data centers and dense wireless networks, using simple deterministic models (dynamical systems). This paper analyzes the approximation error of mean-field models for continuous-time Markov chains (CTMC), and focuses on mean-field models that are represented as finite-dimensional dynamical systems with a unique equilibrium point. By applying Stein's method and the perturbation theory, the paper shows that under some mild conditions, if the mean-field model is globally asymptotically stable and locally exponentially stable, the mean square difference between the stationary distribution of the stochastic system with size M and the equilibrium point of the corresponding mean-field system is O(1/M). The result of this paper establishes a general theorem for establishing the convergence and the approximation error (i.e., the rate of convergence) of a large class of CTMCs to their mean-field limit by mainly looking into the stability of the mean-field model, which is a deterministic system and is often easier to analyze than the CTMCs. Two applications of mean-field models in data center networks are presented to demonstrate the novelty of our results.

This paper investigates the relation between three different notions of privacy: identifiability, differential privacy, and mutual-information privacy. Under a unified privacydistortion framework, where the distortion is defined to be the expected Hamming distance between the input and output databases, we establish some fundamental connections between these three privacy notions. Given a maximum allowable distortion D, we define the privacy-distortion functions ∈i* (D), ∈d*(D), and ∈m*(D) to be the smallest (most private/best) identifiability level, differential privacy level, and mutual information between the input and the output, respectively. We characterize ∈i* (D) and ∈d*(D), and prove that ∈i* (D) - ∈X≤ ∈d*(D) ≤ ∈i* (D) for D within certain range, where ∈Xis a constant determined by the prior distribution of the original database X, and diminishes to zero when X is uniformly distributed. Furthermore, we show that ∈i* (D) and ∈m*(D) can be achieved by the same mechanism for D within certain range, i.e., there is a mechanism that simultaneously minimizes the identifiability level and achieves the best mutual-information privacy. Based on these two connections, we prove that this mutual-information optimal mechanism satisfies ∈-differential privacy with ∈d*(D) ≤ ∈ ≤ ∈d*(D)+2∈X. The results in this paper reveal some consistency between two worst case notions of privacy, namely, identifiability and differential privacy, and an average notion of privacy, mutual-information privacy.

This paper considers the problem of scheduling real-time traffic in wireless networks. We consider ad hoc wireless networks with general conflict graph-based interference model and single-hop traffic. Each packet is associated with a deadline and will be dropped if it is not transmitted before the deadline. The number of packet arrivals in each time-slot and the maximum delay before the deadline are independent and identically distributed across time. We require a minimum fraction of packets to be delivered. At each link, we assume the link keeps track of the difference between the minimum number of packets that need to be delivered so far and the number of packets that are actually delivered, which we call the deficit. The largest-deficit-first (LDF) policy schedules links in descending order according to their deficit values, which is a variation of the longest-queue-first (LQF) policy for non-real-time traffic. We prove that the efficiency ratio of LDF, which is the fraction of the throughput region that LDF can achieve for given traffic distributions, can be lower-bounded by a quantity that we call the real-time local-pooling factor (R-LPF). We further prove that a lower bound on the R-LPF can be related to the weighted sum of the service rates, with a special case of 1/(β+1) by considering the uniform weight, where β is the interference degree of the conflict graph. We also propose a heuristic consensus algorithm that can be used to obtain a good weight vector for such lower bounds for given network topology.

MapReduce/Hadoop framework has been widely used to process large-scale datasets on computing clusters. Scheduling map tasks with data locality consideration is crucial to the performance of MapReduce. Many works have been devoted to increasing data locality for better efficiency. However, to the best of our knowledge, fundamental limits of MapReduce computing clusters with data locality, including the capacity region and theoretical bounds on the delay performance, have not been well studied. In this paper, we address these problems from a stochastic network perspective. Our focus is to strike the right balance between data locality and load balancing to simultaneously maximize throughput and minimize delay. We present a new queueing architecture and propose a map task scheduling algorithm constituted by the Join the Shortest Queue policy together with the MaxWeight policy. We identify an outer bound on the capacity region, and then prove that the proposed algorithm can stabilize any arrival rate vector strictly within this outer bound. It shows that the outer bound coincides with the actual capacity region, and the proposed algorithm is throughput-optimal. Furthermore, we study the number of backlogged tasks under the proposed algorithm, which is directly related to the delay performance based on Little's law. We prove that the proposed algorithm is heavy-traffic optimal, i.e., it asymptotically minimizes the number of backlogged tasks as the arrival rate vector approaches the boundary of the capacity region. Therefore, the proposed algorithm is also delay-optimal in the heavy-traffic regime. The proofs in this paper deal with random processing times with heterogeneous parameters and nonpreemptive task execution, which differentiate our work from many existing works on MaxWeight-type algorithms, so the proof techniques themselves for the stability analysis and the heavy-traffic analysis are also novel contributions.

This paper studies the problem of detecting the information source in a network in which the spread of information follows the popular Susceptible-Infected-Recovered (SIR) model. We assume all nodes in the network are in the susceptible state initially, except one single information source that is in the infected state. Susceptible nodes may then be infected by infected nodes, and infected nodes may recover and will not be infected again after recovery. Given a snapshot of the network, from which we know the graph topology and all infected nodes but cannot distinguish susceptible nodes and recovered nodes, the problem is to find the information source based on the snapshot and the network topology. We develop a sample-path-based approach where the estimator of the information source is chosen to be the root node associated with the sample path that most likely leads to the observed snapshot. We prove for infinite-trees, the estimator is a node that minimizes the maximum distance to the infected nodes. A reverse-infection algorithm is proposed to find such an estimator in general graphs. We prove that for g+1-regular trees such that gq > 1, where g+1 is the node degree and q is the infection probability, the estimator is within a constant distance from the actual source with a high probability, independent of the number of infected nodes and the time the snapshot is taken. Our simulation results show that for tree networks, the estimator produced by the reverse-infection algorithm is closer to the actual source than the one identified by the closeness centrality heuristic. We then further evaluate the performance of the reverse infection algorithm on several real-world networks.

Diffusion processes in networks can be used to model many real-world processes. Analysis of diffusion traces can help us answer important questions such as the source of diffusion and the role of each node in the diffusion process. However, in large-scale networks, it is very expensive if not impossible to monitor the entire network to collect the complete diffusion trace. This paper considers diffusion history reconstruction from a partial observation and develops a greedy, step-by-step reconstruction algorithm. It is proved that the algorithm always produces a diffusion history that is consistent with the partial observation. Our experimental results based on real networks and real diffusion data show that the algorithm significantly outperforms some existing methods.

This paper proposes a pairwise SIR-based random sequential scheduling algorithm for wireless D2D communications. We derive an upper and a lower bound on the number of scheduled links by identifying the equivalence between the proposed algorithm and the Random Sequential Adsorption (RSA) process in physics. We then study the optimal SIR threshold, which is a key parameter in the proposed algorithm, for achieving the maximum sum rate. We finally extend the algorithm when a minimum SIR is required at each scheduled link. From the simulations, we observe that the proposed algorithm can achieve 24% higher sum rate compared with the aggregate SIR-based scheduling algorithm.

The hardware improvement of mobile devices and pervasiveness of wireless technology expedite the convergence with the fast growing cloud computing trend, where the abundant resources on the cloud meet well with the deficiency of hand-held devices. Cloudlet, as a newly emerging paradigm “bringing the cloud closer” to the end users, features a more scalable deployment fashion where idle personal servers can be efficiently harnessed. Despite an envisioned monetary saving, such a paradigm confines itself to limited application scenarios, which fails to reach a wide realm of roaming users outside the distance coverage of the access points. In this paper, we propose a cloudlet-based multi-lateral resource exchange framework for mobile users, relying on no central entities. Inspired by the success of BitCoin, we design a novel virtual currency tailored for our framework. To realize an efficient resource exchange market, we also introduce flexible pricing strategies adopted by the individual users whom we assume are rational price-takers, with solid theoretical analysis on the equilibrium state and the stability. After elaborating the key functional modules, we introduce a prototype design enabling seamless trading among mobile users on Internet bandwidth as a proof-of-concept, with least user intervention. Both simulations and experiments are conducted to verify the practicality and efficiency of our system.

In many computing and networking applications, arriving tasks have to be routed to one of many servers, with the goal of minimizing queueing delays. When the number of processors is very large, a popular routing algorithm works as follows: select two servers at random and route an arriving task to the least loaded of the two. It is well-known that this algorithm dramatically reduces queueing delays compared to an algorithm which routes to a single randomly selected server. In recent cloud computing applications, it has been observed that even sampling two queues per arriving task can be expensive and can even increase delays due to messaging overhead. So there is an interest in reducing the number of sampled queues per arriving task. In this paper, we show that the number of sampled queues can be dramatically reduced by using the fact that tasks arrive in batches (called jobs). In particular, we sample a subset of the queues such that the size of the subset is slightly larger than the batch size (thus, on average, we only sample slightly more than one queue per task). Once a random subset of the queues is sampled, we propose a new load balancing method called batch-filling to attempt to equalize the load among the sampled servers. We show that our algorithm dramatically reduces the sample complexity compared to previously proposed algorithms.

The focus of this paper is on rate and power control algorithms in DSRC. We first design a utility maximization framework by leveraging the well-developed network congestion control, and formulate two subproblems, one on rate control with fixed transmit powers and the other on power control with fixed rates. Distributed rate control and power control algorithms are developed to solve these two subproblems, respectively, and are proved to be asymptotically optimal. Joint rate and power control can be done by using the two algorithms in an alternating fashion. The performance enhancement of our algorithms compared with a recent rate control algorithm, called EMBARC [1], is evaluated by using the network simulator ns2.

We consider ad hoc wireless networks with real-time traffic, and study the capacity requirement of a low-complexity scheduling policy, called largest-deficit-first (LDF), for achieving the same quality of service (QoS) as the optimal policy in networks with unit capacity. We derive theoretical upper and lower bounds for general traffic. The bounds depend on the interference degree of the network and the max/min delay bound ratio. The performance of LDF is further evaluated using simulations and compared with two other algorithms. The simulation results show that LDF significantly outperforms the other two algorithms.

This paper studies the delay constrained multicast capacity of large-scale mobile ad hoc networks (MANETs). We consider a MANET that consists of ns multicast sessions. Each multicast session has one source and p destinations. Each source sends identical information to the p destinations in its multicast session, and the information is required to be delivered to all the p destinations within D time-slots. Assuming the wireless mobiles move according to a 2-D independently and identically distributed mobility model, we first prove that the capacity per multicast session is O(min{1, (log p)(log(nsp))(D/ns)1/2}). We then propose a joint coding/scheduling algorithm achieving a throughput of Θ(min{1, (D/ns)1/2}). Our simulation results suggest that the same scaling law also holds under random walk and random waypoint models.

In this paper, we consider the problem of locating the information source with sparse observations. We assume that a piece of information spreads in a network following a heterogeneous susceptible-infected-recovered (SIR) model, where a node is said to beinfectedwhen it receives the information andrecoveredwhen it removes or hides the information. We further assume that a small subset of infected nodes are reported, from which we need to find the source of the information. We adopt the sample path based estimator developed in [1], and prove that on infinite trees, the sample path based estimator is a Jordan infection center with respect to the set of observed infected nodes. In other words, the sample path based estimator minimizes the maximum distance to observed infected nodes. We further prove that the distance between the estimator and the actual source is upper bounded by a constant independent of the number of infected nodes with a high probability on infinite trees. Our simulations on tree networks and real world networks show that the sample path based estimator is closer to the actual source than several other algorithms.

In standard clustering problems, data points are represented by vectors, and by stacking them together, one forms a data matrix with row or column cluster structure. In this paper, we consider a class of binary matrices, arising in many applications, which exhibit both row and column cluster structure, and our goal is to exactly recover the underlying row and column clusters by observing only a small fraction of noisy entries. We first derive a lower bound on the minimum number of observations needed for exact cluster recovery. Then, we study three algorithms with different running time and compare the number of observations needed by them for successful cluster recovery. Our analytical results show smooth time-data trade offs: one can gradually reduce the computational complexity when increasingly more observations are available.

A vehicular backbone network (VBN) has the potential to augment the Internet with high-throughput data flows for delay-tolerant traffic. High-throughput flows require a joint utilization of transportation capacity for carrying data packets through physical mobility and wireless capacity for switching data packets from one route to another. This paper establishes a model that incorporates both transportation mobility and wireless switching. Then, it characterizes the network capacity based on flow conservation, wireless communication capacity constraints and data storage limits, and solves a convex optimization that results in joint routing and congestion control. A variant with cost minimization reduces delay while maximizing throughput. Next, this paper develops a distributed algorithm that achieves the global objective with limited infrastructure support. Lastly, a packet-level simulation platform using real-world road map and traffic statistics is used to evaluate the distributed algorithm, and demonstrate the significant performance enhancement achieved.

The problem of designing scheduling algorithms for a multichannel (e.g., orthogonal frequency division multiplexing-based) wireless downlink network is considered. The classic MaxWeight algorithm, although throughput-optimal, results in a very poor per-user delay performance in such systems. Hence, an alternate class of algorithms called iterated longest queues first (iLQF) is proposed for overcoming this issue. The iLQF-class algorithms are analyzed in a number of different system configurations. A particular algorithm in this class, called iLQF with pullup, is shown to be rate function optimal for the problem in an appropriate large deviations setting, and is shown to result in a strictly positive value of the rate function for a number of modifications to the basic system model. Thus, the proposed algorithm yields provable performance guarantees. The analytic results are confirmed through simulations.

This paper focuses on scheduling in multihop wireless networks where flows are associated with fixed routes. The well-known back-pressure scheduling algorithm is throughput-optimal, but requires constant exchange of queue length information among neighboring nodes for calculating the “back-pressure.” Moreover, previous research shows that the total queue length along a route increases quadratically as the route length under the back-pressure algorithm, resulting in poor delay performance. In this paper, we propose a self-regulated MaxWeight scheduling, which does not require back-pressure calculation. We prove that the self-regulated MaxWeight scheduling is throughput-optimal (an algorithm is said to be throughput-optimal if it can stabilize any traffic that can be stabilized by any other algorithm). In the simulation part, we show that the self-regulated MaxWeight scheduling has a much better delay performance than the back-pressure algorithm.

We consider the problem of distributed admission control without knowledge of the capacity region in single-hop wireless networks, for flows that require a pre-specified bandwidth from the network. We present an optimization framework that allows us to design a scheduler and resource allocator, and by properly choosing a suitable utility function in the resource allocator, we prove that existing flows can be served with a prespecified bandwidth, while the link requesting admission can determine the largest rate that it can get such that it does not interfere with the allocation to the existing flows.

Scheduling map tasks to improve data locality is crucial to the performance of MapReduce. Many works have been devoted to increasing data locality for better efficiency. However, to the best of our knowledge, fundamental limits of MapReduce computing clusters with data locality, including the capacity region and theoretical bounds on the delay performance, have not been studied. In this paper, we address these problems from a stochastic network perspective. Our focus is to strike the right balance between data-locality and load-balancing to simultaneously maximize throughput and minimize delay. We present a new queueing architecture and propose a map task scheduling algorithm constituted by the Join the Shortest Queue policy together with the MaxWeight policy. We identify an outer bound on the capacity region, and then prove that the proposed algorithm stabilizes any arrival rate vector strictly within this outer bound. It shows that the algorithm is throughput optimal and the outer bound coincides with the actual capacity region. Further, we study the number of backlogged tasks under the proposed algorithm, which is directly related to the delay performance based on Little's law. We prove that the proposed algorithm is heavy-traffic optimal, i.e., it asymptotically minimizes the number of backlogged tasks as the arrival rate vector approaches the boundary of the capacity region. Therefore, the proposed algorithm is also delay optimal in the heavy-traffic regime.

The rapid advancement of smartphones has instigated tremendous data applications for cell phones. Supporting simultaneous voice and data services in a cellular network is not only desirable but also becoming indispensable. However, if the voice and data are serviced through the same antenna (like the 3G UMTS network), a voice call with data sessions requires better radio connection than a voice-only call. In this paper, we systematically study the coordination between the voice and data transmissions in UMTS networks. From analyzing a large carrier's UMTS network recording data, we first identify the most relevant network measurements/features indicating a potential call drop, then propose a drop-call predictor based on AdaBoost. Moreover, we develop an intelligent call management strategy to voluntarily block data sessions when the voice is predicted to be dropped. Our analysis utilizing real service provider's data sets shows that our proposed scheme can not only predict drop calls with a very high accuracy but also achieve the highest user satisfaction compared to the other existing call management strategies.

This paper considers the problem of scheduling real-time traffic in wireless networks. We consider an ad hoc wireless network with general interference and general one-hop traffic. Each packet is associated with a deadline and will be dropped if it is not transmitted before the deadline expires. The number of packet arrivals in each time slot and the length of a deadline are both stochastic and follow certain distributions. We only allow a fraction of packets to be dropped. At each link, we assume the link keeps track of the difference between the minimum number of packets that need to be delivered and the number of packets that are actually delivered, which we call deficit. The largest-deficit-first (LDF) policy schedules links in descending order according to their deficit values, which is a variation of the largest-queue-first (LQF) policy for non-real-time traffic. We prove that the efficiency ratio of LDF can be lower bounded by a quantity that we call the real-time local-pooling factor (R-LPF). We further prove that given a network with interference degree β, the R-LPF is at least 1/(β+1), which in the case of the one-hop interference model translates into an R-LPF of at least 1/3.

This paper studies the allocation of feedback resources in the downlink of a frequency-division duplex (FDD) multichannel wireless system. We consider a downlink network with a single base station, L shared channels, and N mobile users. Throughput optimal algorithms like MaxWeight in general require the complete channel-state information (CSI) ( NL link states) for scheduling. Acquiring the complete CSI, however, is a prohibitive overhead in multichannel networks when the number of users is large. In this paper, we consider the scenario where the base station allocates only a limited amount of uplink resources for acquiring channel-state information. We first show that to support a (1-ε) fraction of the full throughput region (the throughput region with the complete CSI), the base station needs to acquire at least Θ((1-ε)L) link states at each time-slot. We then propose a Weight-Based Feedback allocation, named WBF, and show that WBF together with MaxWeight scheduling achieves a (1-ε) fraction of the full throughput region by acquiring Θ(L log[1/(ε)]) link states per time-slot. For i.i.d. on-off channels, we further prove that Θ(Llog[1/(ε)]) link states per time-slot is necessary for achieving a (1-ε) fraction of the full throughput region.

Cloud computing services are becoming ubiquitous, and are starting to serve as the primary source of computing power for both enterprises and personal computing applications. We consider a stochastic model of a cloud computing cluster, where jobs arrive according to a stochastic process and request virtual machines (VMs), which are specified in terms of resources such as CPU, memory and storage space. While there are many design issues associated with such systems, here we focus only on resource allocation problems, such as the design of algorithms for load balancing among servers, and algorithms for scheduling VM configurations. Given our model of a cloud, we first define its capacity, i.e., the maximum rates at which jobs can be processed in such a system. Then, we show that the widely-used Best-Fit scheduling algorithm is not throughput-optimal, and present alternatives which achieve any arbitrary fraction of the capacity region of the cloud. We then study the delay performance of these alternative algorithms through simulations.

In wireless networks, the transmission rate of a link is determined by received signal strength, interference from simultaneous transmissions, and available coding-modulation schemes. Rate allocation is a key problem in wireless network design, but a very challenging problem because: (i) wireless interference is global, i.e., a transmission interferes all other simultaneous transmissions, and (ii) the rate-power relation is non-convex and non-continuous, where the discontinuity is due to limited number of coding-modulation choices in practical systems. In this paper, we consider a realistic Signal-to-Interference-and-Noise-Ratio (SINR) based interference model, and assume continuous power space and finite rate options (coding-modulation choices). We propose a distributed power control and coding-modulation adaptation algorithm using annealed Gibbs sampling, which achieves throughput optimality in an arbitrary network topology.

An abstraction of the physical-layer coding using bit pipes that are coupled through data-rates is insufficient to capture notions such as node cooperation in cooperative relay networks. Consequently, network-stability analyses based on such abstractions are valid for non-cooperative schemes alone and meaningless for cooperative schemes. Motivated from this, this paper develops a framework that brings the information-theoretic coding scheme together with network-stability analysis. This framework does not constrain the system to any particular achievable scheme, i.e., the relays can use any cooperative coding strategy of its choice such as amplify/compress/quantize or any alter-and-forward scheme. The paper focuses on the scenario when coherence duration is of the same order of the packet/codeword duration, the channel distribution is unknown and the fading state is only known causally. The main contributions of this paper are two-fold: first, it develops a low-complexity queue-architecture to enable stable operation of cooperative relay networks, and, second, it establishes the throughput optimality of a simple network algorithm that utilizes this queue-architecture.

This paper considers the problem of designing scheduling algorithms for multichannel (e.g., OFDM-based) wireless downlink networks, with a large number of users and proportionally large bandwidth. For this system, while the classical MaxWeight algorithm is known to be throughput-optimal, its buffer-overflow performance is very poor (formally, it is shown that it has zero rate function in our setting). To address this, a class of algorithms called iterated Heaviest matching with Longest Queues First (iHLQF) is proposed. The algorithms in this class are shown to be throughput-optimal for a general class of arrival/channel processes, and also rate-function-optimal (i.e., exponentially small buffer overflow probability) for certain arrival/channel processes. iHLQF, however, has higher complexity than MaxWeight (n4versusn2, respectively). To overcome this issue, a new algorithm called Server-Side Greedy (SSG) is proposed. It is shown that SSG is throughput-optimal, results in a much better per-user buffer overflow performance than the MaxWeight algorithm (positive rate function for certain arrival/channel processes), and has a computational complexity (n2) that is comparable to the MaxWeight algorithm. Thus, it provides a nice tradeoff between buffer-overflow performance and computational complexity. These results are validated by both analysis and simulations.

We study an intermittently connected network (ICN) composed of multiple clusters of wireless nodes. Within each cluster, nodes can communicate directly using the wireless links. However, these clusters are far away from each other such that direct communication between the clusters is impossible except through “mobile” contact nodes. These mobile contact nodes are data carriers that shuffle between clusters and transport data from the source to the destination clusters. There are several applications of our network model, such as clusters of mobile soldiers connected via unmanned aerial vehicles. Our work here focuses on a queue-based cross-layer technique known as the back-pressure algorithm. The algorithm is known to be throughput-optimal, as well as resilient to disruptions in the network, making it an ideal candidate communication protocol for our intermittently connected network. In this paper, we design a back-pressure routing/rate control algorithm for ICNs. Though it is throughput-optimal, the back-pressure algorithm has several drawbacks when used in ICNs, including long end-to-end delays, large number of potential queues needed, and loss in throughput due to intermittency. We present a modified back-pressure algorithm that addresses these issues. We implement our algorithm on a 16-node experimental testbed and present our experimental results in this paper.

The rapid increase of content delivery over the Internet has led to the proliferation of content distribution networks (CDNs). Management of CDNs requires algorithms for request routing, content placement, and eviction in such a way that user delays are small. We abstract the system of frontend source nodes and backend caches of the CDN in the likeness of the input and output nodes of a switch. In this model, queues of requests for different pieces of content build up at the source nodes, which route these requests to a cache that contains the requested content. For each request that is routed to a cache, a corresponding data file is transmitted back to the requesting source across links of finite capacity. Caches are of finite size, and the content of the caches can be refreshed periodically. Our objective is to design policies for request routing, content placement and content eviction with the goal of small user delays. Stable policies ensure the finiteness of the request queues, while good polices also lead to short queue lengths. We first design a throughput-optimal algorithm that solves the routing-placement-eviction problem. The design yields insight into the impact of different cache refresh policies on queue length, and we construct throughput optimal algorithms that engender short queue lengths. We illustrate the potential of our approach through simulations on different CDN topologies.

This paper considers the problem of designing scheduling algorithms for multi-channel (e.g., OFDM-based) wireless downlink systems. We show that the Server-Side Greedy (SSG) rule introduced in earlier papers for ON-OFF channels performs well even for more general channel models. The key contribution in this paper is the development of new mathematical techniques for analyzing Markov chains that arise when studying general channel models. These techniques include a way of calculating the distribution of the maximum of a multi-dimensional Markov chain (note that the maximum does not have the Markov property on its own), and also a Markov chain stochastic dominance result using coupling arguments.

MaxWeight scheduling has gained enormous popularity as a powerful paradigm for achieving queue stability and maximum throughput in a wide variety of scenarios. The maximum-stability guarantees however rely on the fundamental premise that the system consists of a fixed set of flows with stationary ergodic traffic processes. In the present paper we examine networks where the population of active flows varies over time, as flows eventually end while new flows occasionally start. We show that MaxWeight policies may fail to provide maximum stability due to persistent inefficient spatial reuse. The intuitive explanation is that these policies tend to serve flows with large backlogs, even when the resulting spatial reuse is not particularly efficient, and fail to exploit maximum spatial reuse patterns involving flows with smaller backlogs. These results indicate that instability of MaxWeight scheduling can occur due to spatial inefficiency in networks with fixed transmission rates, which is fundamentally different from the inability to fully exploit time-varying rates shown in prior work. We discuss how the potential instability effects can be countered by spatial traffic aggregation, and describe some of the associated challenges and performance trade-offs.

This paper studies the problem of scheduling in single-hop wireless networks with real-time traffic, where every packet arrival has an associated deadline and a minimum fraction of packets must be transmitted before the end of the deadline. Using optimization and stochastic network theory we study the problem of scheduling to meet quality of service (QoS) requirements under heterogeneous delay constraints and time-varying channel conditions. Our analysis results in an optimal scheduling algorithm which fairly allocates data rates to all flows while meeting long-term delay demands. We also prove that under a simplified scenario our solution translates into a greedy strategy that makes optimal decisions with low complexity.

In this paper, we consider a network of stationary nodes that rely on mobile nodes to transport data between them. We assume the mobile nodes can control their mobility pattern to respond to traffic loads, as well as satisfy some other secondary objectives, such as surveillance requirements. We study this problem in the framework of cost minimization, where we derive a dual iterative algorithm that results in optimal mobility pattern for minimizing network wide cost. We then implement our proposed algorithm and evaluate its performance on a testbed.

The growth of real-time content streaming over the Internet has resulted in the use of peer-to-peer (P2P) approaches for scalable content delivery. In such P2P streaming systems, each peer maintains a playout buffer of content chunks which it attempts to fill by contacting other peers in the network. The objective is to ensure that the chunk to be played out is available with high probability while keeping the buffer size small. A small playout buffer means that the playout delay is small. Thus, the objective is to study the tradeoff between two measures of QoS, chunk playout rate and delay. A policy is a rule that suggests which chunks should be requested by the peer from other peers. We consider consider a number of recently suggested policies consistent with buffer minimization for a given target of skip-free playout. We first study a rarest-first policy that attempts to obtain chunks farthest from playout, and a greedy policy that attempts to obtain chunks nearest to playout. We show that they both have similar buffer scalings (as a function of the number of peers of target probability of skip-free probability). We then study a hybrid policy which achieves order sense improvements over both policies and can achieve order optimal performance. We validate our results using simulations.

A wireless broadcast network model with secrecy constraints is investigated, in which a source node broadcastsKconfidential message flows toKuser nodes, with each message intended to be decoded accurately by one user and to be kept secret from all other users (who are thus considered to be eavesdroppers with regard to all other messages but their own). The source maintains a queue for each message flow if it is not served immediately. The channel from the source to theKusers is modeled as a fading broadcast channel, and the channel state information is assumed to be known to the source and the corresponding receivers. Two eavesdropping models are considered. For a collaborative eavesdropping model, in which the eavesdroppers exchange their outputs, the secrecy capacity region is obtained, within which each rate vector is achieved by using a time-division scheme and a source power control policy over channel states. A throughput optimal queue-length-based rate scheduling algorithm is further derived that stabilizes all arrival rate vectors contained in the secrecy capacity region. Moreover, the network utility function is maximized via joint design of rate control, rate scheduling, power control, and secure coding. More precisely, a source controls the message arrival rate according to its message queue, the rate scheduling selects a transmission rate based the queue length vector, and the rate vector is achieved by power control and secure coding. These components work jointly to solve the network utility maximization problem. For a noncollaborative eavesdropping model, in which eavesdroppers do not exchange their outputs, an achievable secrecy rate region is derived based on a time-division scheme, and the queue-length-based rate scheduling algorithm and the corresponding power control policy are obtained that stabilize all arrival rate vectors in this region. The network utility maximizing rate control vector is also obtained.

The secrecy throughput of mobile ad hoc networks (MANETs) with malicious nodes is investigated. The MANET consists ofnlegitimate mobile nodes andmmalicious nodes. Transmissions between legitimate nodes are subject to a delay constraintD. A model under passive attack is first studied, in which the malicious nodes are assumed to be eavesdroppers that only listen to transmission without actively injecting signals. An information-theoretic approach for security is applied to achieve secure communication among legitimate nodes in MANETs with transmissions being kept perfectly secure from eavesdroppers. A critical threshold on the number of malicious nodes (m) is identified such that whenm=o(√{nD}), i.e., limn→∞m/√{nD} = 0, the optimal secrecy throughput equals that of MANETs without malicious nodes, i.e., the impact of the presence of malicious nodes on the network throughput is negligible; and whenm= Ω(√{nD}poly(n)), i.e., limn→∞m/(√{nD}poly(n)) ≥cfor a positive constant c, the optimal secrecy throughput is limited by the number of malicious nodes. A model under active attack is further studied, in which the malicious nodes actively attack the network by transmitting modified packets to the destination nodes. It is shown that to guarantee the same throughput as the model under passive attack, the model under active attack needs to satisfy more stringent condition on the number of malicious nodes.

The problem of routing/scheduling in a wireless network with partial/delayed network (channel and queue) state information (NSI) is studied in this paper. Two cases are considered: (i) centralized routing/scheduling, where a central controller obtains heterogeneously delayed information from each of the nodes (thus, the controller has NSI with different delays from different nodes), and makes routing/scheduling decisions; (ii) decentralized routing/scheduling, where each node makes a decision based on its current channel and queue states along with homogeneous delayed NSI from other nodes. For each of the cases (with additional flow restrictions for the decentralized routing/scheduling case), the optimal network throughput regions are characterized under the above described NSI models and it is shown that the throughput regions shrinks with the increase of delay. Further, channel and queue length based routing/scheduling algorithms that achieve the above throughput regions are proposed in this paper.

We study the optimal control of communication networks in the presence of heterogeneous traffic requirements. Specifically, we distinguish the flows into two crucial classes: inelastic for modeling high-priority, delay-sensitive, and fixed-throughput applications; and elastic for modeling low-priority, delay-tolerant, and throughput-greedy applications. We note that the coexistence of such diverse flows creates complex interactions at multiple levels (e.g., flow and packet levels), which prevent the use of earlier design approaches that dominantly assume homogeneous traffic. In this work, we develop the mathematical framework and novel design methodologies needed to support such heterogeneous requirements and propose provably optimal network algorithms that account for the multilevel interactions between the flows. To that end, we first formulate a network optimization problem that incorporates the above throughput and service prioritization requirements of the two traffic types. We, then develop a distributed joint load-balancing and congestion control algorithm that achieves the dual goal of maximizing the aggregate utility gained by the elastic flows while satisfying the fixed throughput and prioritization requirements of the inelastic flows. Next, we extend our joint algorithm in two ways to further improve its performance: in delay through a virtual queue implementation with minimal throughput degradation and in utilization by allowing for dynamic multipath routing for elastic flows. A unique characteristic of our proposed dynamic routing solution is the novel two-stage queueing architecture it introduces to satisfy the service prioritization requirement.

We consider multiuser scheduling in wireless networks with channel variations and flow-level dynamics. Recently, it has been shown that the MaxWeight algorithm, which is throughput-optimal in networks with a fixed number of users, fails to achieve the maximum throughput in the presence of flow-level dynamics. In this paper, we propose a new algorithm, called Workload-based Scheduling with Learning, which is provably throughput-optimal, requires no prior knowledge of channels and user demands, and performs significantly better than previously suggested algorithms.

Back-pressure-type algorithms based on the algorithm by Tassiulas and Ephremides have recently received much attention for jointly routing and scheduling over multihop wireless networks. However, this approach has a significant weakness in routing because the traditional back-pressure algorithm explores and exploits all feasible paths between each source and destination. While this extensive exploration is essential in order to maintain stability when the network is heavily loaded, under light or moderate loads, packets may be sent over unnecessarily long routes, and the algorithm could be very inefficient in terms of end-to-end delay and routing convergence times. This paper proposes a new routing/scheduling back-pressure algorithm that not only guarantees network stability (throughput optimality), but also adaptively selects a set of optimal routes based on shortest-path information in order to minimize average path lengths between each source and destination pair. Our results indicate that under the traditional back-pressure algorithm, the end-to-end packet delay first decreases and then increases as a function of the network load (arrival rate). This surprising low-load behavior is explained due to the fact that the traditional back-pressure algorithm exploits all paths (including very long ones) even when the traffic load is light. On the other-hand, the proposed algorithm adaptively selects a set of routes according to the traffic load so that long paths are used only when necessary, thus resulting in much smaller end-to-end packet delays as compared to the traditional back-pressure algorithm .

The back-pressure algorithm introduced in 1992 by Tassiulas and Ephremides is a well-known distributed and adaptive routing/scheduling algorithm where nodes only need the queue-length information of neighboring nodes to make routing decisions. Packets are adaptively routed in the network according to congestion information, which makes the algorithm resilient to traffic and topology changes. However, the back-pressure algorithm requires routers to maintain a separate queue for each destination, which precludes its implementation in large-scale networks. In this paper, we propose a distributed cluster-based back-pressure routing algorithm that retains the adaptability of back-pressure routing while significantly reducing the number of queues that have to be maintained at each node.

This paper considers the problem of designing scheduling algorithms for multi-channel (e.g., OFDM) wireless downlink networks with n users/OFDM sub-channels. For this system, while the classical MaxWeight algorithm is known to be throughput-optimal, its buffer-overflow performance is very poor (formally, we show it has zero rate function in our setting). To address this, we propose a class of algorithms called iHLQF (iterated Heaviest matching with Longest Queues First) that is shown to be throughput optimal for a general class of arrival/channel processes, and also rate-function optimal (i.e., exponentially small buffer overflow probability) for certain arrival/channel processes. iHLQF however has higher complexity than MaxWeight (n4vs. n2respectively). To overcome this issue, we propose a new algorithm called SSG (Server-Side Greedy). We show that SSG is throughput optimal, results in a much better per-user buffer overflow performance than the MaxWeight algorithm (positive rate function for certain arrival/channel processes), and has a computational complexity (n2) that is comparable to the MaxWeight algorithm. Thus, it provides a nice trade-off between buffer-overflow performance and computational complexity. These results are validated by both analysis and simulations.

We consider multiuser scheduling in wireless networks with channel variations and flow-level dynamics. Recently, it has been shown that the MaxWeight algorithm, which is throughput-optimal in networks with a fixed number users, fails to achieve the maximum throughput in the presence of flow-level dynamics. In this paper, we propose a new algorithm, calledworkload-based scheduling with learning, which is provably throughput-optimal, requires no prior knowledge of channels and user demands, and performs significantly better than previously suggested algorithms.

We study a mobile wireless network where groups or clusters of nodes are intermittently connected via mobile "carriers'' (the carriers provide connectivity over time among different clusters of nodes). Over such networks (an instantiation of a delay tolerant network), it is well-known that traditional routing algorithms perform very poorly. In this paper, we propose a two-level Back- Pressure with Source-Routing algorithm (BP+SR) for such networks. The proposed BP+SR algorithm separates routing and scheduling within clusters (fast time-scale) from the communications that occur across clusters (slow time-scale), without loss in network throughput (i.e., BP+SR is throughput-optimal). More importantly, for a source and destination node that lie in different clusters, the traditional back-pressure algorithm results in large queue lengths at each node along its path. This is because the queue dynamics are driven by the slowest time-scale (i.e., that of the carrier nodes) along the path between the source and destination, which results in very large end-to-end delays. On the other-hand, we show that the two-level BP+SR algorithm maintains large queues only at a very few nodes, and thus results in order-wise smaller end-to-end delays. We provide analytical as well as simulation results to confirm our claims.

While there has been much progress in designing backpressure based stabilizing algorithms for multihop wireless networks, end-to-end performance (e.g., end-to-end buffer usage) results have not been as forthcoming. In this paper, we study the end-to-end buffer usage (sum of buffer utilization along a flow path) over a network with general topology and with fixed, loop-free routes using a large-deviations approach. We first derive bounds on the best performance that any scheduling algorithm can achieve. Based on the intuition from the bounds, we propose a class of (backpressure-like) scheduling algorithms called ¿ß-algorithms. We show that the parameters ¿ and ß can be chosen such that the system under the ¿ß-algorithm performs arbitrarily closely to the best possible scheduler (formally the decay rate function for end-to-end buffer overflow is shown to be arbitrarily close to optimal in the large-buffer regime). We also develop variants which have the same asymptotic optimality property, and also provide good performance in the small-buffer regime. Our results are substantiated using both analysis and simulation.

This paper studies the delay constrained multicast capacity of large scale mobile ad hoc networks (MANETs). We consider a MANET that consists of nsmulticast sessions. Each multicast session has one source and p destinations. Each source sends identical information to the p destinations in its multicast session, and the information is required to be delivered to all the p destinations within D time-slots. Assuming the wireless mobiles move according to a two-dimensional i.i.d. mobility model, we first prove that the capacity per multicast session is O(min{1, (log p)(log (nsp)) ¿(D/ns)}). We then propose a joint coding/scheduling algorithm achieving a throughput of ¿ (min {1, ¿(D/ns)}). Our simulation results suggest that the same scaling law also holds under random walk and random waypoint models.

This paper studies the delay, cost and infrastructure tradeoff of epidemic routing in mobile sensor networks. We consider a mobile sensor network with M mobiles and B static base stations. The mobile sensors collect information when moving around and need to report the information to the base stations. Three different epidemic routing schemes --- target epidemic routing, uncontrolled epidemic routing and controlled epidemic routing --- are analyzed in this paper. For each of the three schemes, we characterize the scaling behaviors of the delay, which is defined to be the average number of time slots required to deliver a message, and the cost, which is defined to be the average number of transmissions required to deliver a message, in terms of the number of mobiles (M) and the number of base stations (B). These scaling results reveal the fundamental tradeoff among delay, cost and infrastructure in mobile sensor networks.

This paper studies feedback resource allocation in the downlink of a Frequency Division Duplex (FDD) multichannel wireless system. We consider a downlink network with a single base station, L shared channels and N mobile users. Throughput optimal algorithms like the MaxWeight scheduling in general require the complete channel state information (with N × L link states) for scheduling, which could be unaffordably expensive when the number of users is large. In this paper, we consider the scenario where the base station allocates only limited uplink resource for acquiring channel state information. We first show that to support a fraction (1 - ε) of the full throughput region (the throughput region with full channel state information), the base station needs to acquire at least Θ(L) link states at each time slot. We then propose a Weight Based Feedback allocation, named WBF, and show that WBF together with the MaxWeight scheduling achieves a fraction (1 - ε) of the full throughput region by acquiring at most Θ(L log 1 ⁄ ε) link states per time slot.

This paper studies scheduling in multichannel wireless networks with flow-level dynamics. We consider a downlink network with a single base station, M channels (frequency bands), and multiple mobile users (flows). We also assume mobiles dynamically join the network to receive finite-size files and leave after downloading the complete files. A recent study [16] has shown that the MaxWeight algorithm fails to be throughput-optimal under this flow-level dynamics. The main contribution of this paper is the development of joint channel-assignment and workload-based scheduling algorithms for multichannel downlink networks with dynamic flow arrivals/departures. We prove that these algorithms are throughput-optimal. Our simulations further demonstrate that a hybrid channel-assignment and workload-based scheduling algorithm significantly improves the network performance (in terms of both file-transfer delay and blocking probability) compared to the existing algorithms.

In this work, we study the control of communication networks in the presence of both inelastic and elastic traffic flows. The characteristics of these two types of traffic differ significantly. Hence, earlier approaches that focus on homogeneous scenarios with a single traffic type are not directly applicable. We formulate a new network optimization problem that incorporates the performance requirements of inelastic and elastic traffic flows. The solution of this problem provides us with a new queueing architecture, and distributed load balancing and congestion control algorithm with provably optimal performance. In particular, we show that our algorithm achieves the dual goal of maximizing the aggregate utility gained by the elastic flows while satisfying the demands of inelastic flows. Our base optimal algorithm is extended to provide better delay performance for both types of traffic with minimal degradation in throughput. It is also extended to the practically relevant case of dynamic arrivals and departures. Our solution allows for a controlled interaction between the performance of inelastic and elastic traffic flows. This performance can be tuned to achieve the appropriate design tradeoff. The network performance is studied both theoretically and through extensive simulations.

We study throughput-optimal scheduling/routing over mobile ad-hoc networks with time-varying (fading) channels. Traditional back-pressure algorithms (based on the work by Tassiulas and Ephremides) require instantaneous network state (topology, queues-lengths, and fading channel-state) in order to make scheduling/routing decisions. However, such instantaneous network-wide (global) information is hard to come by in practice, especially when mobility induces a time-varying topology. With information delays and a lack of global network state, different mobile nodes have differing "views" of the network, thus inducing uncertainty and inconsistency across mobile nodes in their topology knowledge and network state information. In such a setting, we first characterize the through-optimal rate region and develop a back-pressure-like scheduling algorithm, which we show is throughput-optimal. Then, by partitioning the geographic region spatially into disjoint tiles, and sharing delayed topology and network state information only among mobile nodes currently within each tile, we develop a localized low-complexity scheduling algorithm. The algorithm uses instantaneous local information (the queue length, channel state and current position at a mobile node) along with delayed network state information from nodes that were within its tile (i.e., from nodes that were within a nearby geographic region as opposed to network-wide information). The proposed algorithm is shown to be near-optimal, where the geographic distance over which delayed network-state information is shared determines the provable lower bound on the achievable throughput.

Back-pressure based algorithms based on the algorithm by Tassiulas and Ephremides have recently received much attention for jointly routing and scheduling over multi-hop wireless networks. However a significant weakness of this approach has been in routing, because the traditional back-pressure algorithm explores and exploits all feasible paths between each source and destination. While this extensive exploration is essential in order to maintain stability when the network is heavily loaded, under light or moderate loads, packets may be sent over unnecessarily long routes and the algorithm could be very inefficient in terms of end-to-end delay and routing convergence times. This paper proposes new routing/scheduling back-pressure algorithms that not only guarantees network stability (through-put optimality), but also adaptively selects a set of optimal routes based on shortest-path information in order to minimize average path-lengths between each source and destination pair. Our results indicate that under the traditional back-pressure algorithm, the end-to-end packet delay first decreases and then increases as a function of the network load (arrival rate). This surprising low-load behavior is explained due to the fact that the traditional back-pressure algorithm exploits all paths (including very long ones) even when the traffic load is light. On the otherhand, the proposed algorithm adaptively selects a set of routes according to the traffic load so that long paths are used only when necessary, thus resulting in much smaller end-to-end packet delays as compared to the traditional back-pressure algorithm.

The secrecy throughput of mobile ad-hoc networks (MANETs) with malicious nodes is investigated. The MANET consists of n legitimate mobile nodes and m malicious nodes. Transmissions between legitimate nodes are subject to a delay constraint D. An information theoretic approach for security is applied to achieve secure communication among legitimate nodes in MANETs with transmissions being kept perfectly secure from malicious nodes. A critical threshold on the number of malicious nodes (m) is identified such that when m = o(radicnD), i.e., limnrarrinfinm/radicnD = 0, the secrecy throughput equals the throughput of MANETs without malicious nodes, i.e., the impact of the presence of malicious nodes on the network throughput is negligible; and when m = Omega (radicnDpoly(n)), i.e., limnrarrinfinm/(radicnDpoly(n)) ges c for a positive constant c, the secrecy throughput is limited by the number of malicious nodes.

Recent advances in computer technology and wireless communications have enabled the emergence of stream-based sensor networks. In such sensor networks, real-time data are generated by a large number of distributed sources. Queries are made that may require sophisticated processing and filtering of the data. A query is represented by a query graph. In order to reduce the data transmission and to better utilize resources, it is desirable to place operators of the query graph inside the network, and thus to perform in-network processing. Moreover, given that various queries occur with different frequencies and that only a subset of sensor data may actually be queried, caching intermediate data objects inside the network can help improve query efficiency. In this paper, we consider the problem of placing both operators and intermediate data objects inside the network for a set of queries so as to minimize the total cost of storage, computation, and data transmission. We propose distributed algorithms that achieve optimal solutions for tree-structured query graph topologies and general network topologies. The algorithms converge in Lmax(.HQ+ 1) iterations, where Lmaxis the order of the diameter of the sensor network, and Hq represents the depth of the query graph, defined as the maximum number of operations needed for a raw data to become a final data. For a regular grid network and complete binary tree query graph, the complexity is 0(radic(N)log2M), where N is the number of nodes in the sensor network and M is the number of data objects in a query graph. The most attractive features of these algorithms are that they require only information exchanges between neighbors, can be executed asynchronously, are adaptive to cost change and topology change, and are resilient to node or link failures.

We study scalable, distributed, and adaptive routing algorithms for communication networks. The back-pressure algorithm introduced in [21] is a well-known distributed and adaptive routing/scheduling algorithm where nodes only need the queue length information of neighboring nodes to make routing decisions, and packets are adaptively routed in the network according to congestion information, which makes the algorithm resilient to traffic and topology changes. However, the back-pressure algorithm requires routers to maintain a separate queue for each destination, which prevents its implementation in large-scale networks like the Internet. In this paper, we propose a cluster-based back-pressure routing algorithm, which retains the distributability and adaptability of back-pressure routing, while significantly reducing the number of queues that have to be maintained at each node. Since the cluster-based algorithm performs adaptive load-balancing in the network, it has the potential to eliminate the need for off-line traffic engineering in the Internet.

In this paper, we investigate the delay–throughput tradeoffs in mobilead-hocnetworks. We consider four node mobility models: 1) two-dimensional independent and identically distributed (i.i.d.) mobility, 2) two-dimensional hybrid random walk, 3) one-dimensional i.i.d. mobility, and 4) one-dimensional hybrid random walk. Two mobility time scales are included in this paper. i) Fast mobility, where node mobility is at the same time scale as data transmissions. ii) Slow mobility, where node mobility is assumed to occur at a much slower time scale than data transmissions. Given a delay constraint$D$, we first characterize the maximum throughput per source–destination (S-D) pair for each of the four mobility models with fast or slow mobiles. We then develop joint coding–scheduling algorithms to achieve the optimal delay–throughput tradeoffs.

In this correspondence, we consider a wireless sensor network consisting of n sensors, and each sensor has a measurement, which is an integer value belonging to the set {().....m-1}, so that it can be represented by [log2m] bits. The network has a special node called the fusion center whose goal is to compute a symmetric function of these measurements. The problem studied is to minimize the total transmission energy used by the network when computing this function, subject to the constraint that this computation be correct with high probability. We assume the wireless channels are binary symmetric channels with a probability of error p, and that each sensor uses ralphaunits of energy to transmit each bit, where r is the transmission range of the sensor.

We consider a cellular network consisting of a base station and N receivers. The channel states of the receivers are assumed to be identical and independent of each other. The goal is to compare the throughput of two different scheduling policies ( a queue-length-based policy and a greedy scheduling policy) given an upper bound on the queue overflow probability. We consider a multi-state channel model, where each channel is assumed to be in one of L states. Given an upper bound on the queue overflow probability, we obtain a lower bound on the throughput of the queue-length-based policy. For sufficiently large N, the lower bound is shown to be tight, strictly increasing with N, and strictly larger than the throughput of the greedy policy

We consider a wireless sensor network consisting of n sensors, each having a recorded bit, the sensor’s measurement, which has been set to either “0” or “1”. The network has a special node called the fusion center whose goal is to compute a symmetric function of these bits; i.e., a function that depends only on the number of sensors that have a “1.” The sensors convey information to the fusion center in a multi-hop fashion to enable the function computation. The problem studied is to minimize the total transmission energy used by the network when computing this function, subject to the constraint that this computation is correct with high probability. We assume the wireless channels are binary symmetric channels with a probability of error p, and that each sensor uses rαunits of energy to transmit each bit, where r is the transmission range of the sensor. The main result in this paper is an algorithm whose energy usage is Θ (n(loglogn)(√logn/n)α), and we also show that any algorithm satisfying the performance constraints must necessarily have energy usage Ω (n(√logn/n)α). Then, we consider the case where the sensor network observes N events, and each node records one bit per event, thus having N bits to convey. The fusion center now wants to compute N symmetric functions, one for each of the events.

In this correspondence, we consider a cellular network consisting of a base station and N receivers. The channel states of the receivers are assumed to be identical and independent of each other. The goal is to compare the throughput of two different scheduling policies (a queue-length-based (QLB) policy and a greedy policy) given an upper bound on the queue overflow probability or the delay violation probability. We consider a multistate channel model, where each channel is assumed to be in one of L states. Given an upper bound on the queue overflow probability or an upper bound on the delay violation probability, we show that the total network throughput of the (QLB) policy is no less than the throughput of the greedy policy for all N. We also obtain a lower bound on the throughput of the (QLB) policy. For sufficiently large N, the lower bound is shown to be tight, strictly increasing with N, and strictly larger than the throughput of the greedy policy. Further, for a simple multistate channel model-ON-OFF channel, we prove that the lower bound is tight for all N

Phase unwrapping is an important problem in many magnetic resonance imaging applications, such as field mapping and flow imaging. The challenge in two-dimensional phase unwrapping lies in distinguishing jumps due to phase wrapping from those due to noise and/or abrupt variations in the actual function. This paper addresses this problem using a Markov random field to model the true phase function, whose parameters are determined by maximizing the a posteriori probability. To reduce the computational complexity of the optimization procedure, an efficient algorithm is also proposed for parameter estimation using a series of dynamic programming connected by the iterated conditional modes. The proposed method has been tested with both simulated and experimental data, yielding better results than some of the state-of-the-art method (e.g., the popular least-squares method) in handling noisy phase images with rapid phase variations.

We consider the problem of allocating resources at a base station to many competing flows, when each flow is intended for a different receiver. The channel conditions may be time-varying and different for different receivers. It has been shown in A. Eryilmaz and R. Srikant (2005) that in a delay-free network, a combination of queue-length-based scheduling at the base station and congestion control at the end users can guarantee queue-length stability and fair resource allocation. In this paper, we extend this result to wireless networks where the congestion information from the base station is received with a feedback delay at the transmitters. The delays can be heterogeneous (i.e., different users may have different round-trip delays) and time-varying, but are assumed to be upper-bounded, with possibly very large upper bounds. We show that the joint congestion control-scheduling algorithm continues to be stable and continues to provide a fair allocation of the network resources.

When estimating terrain elevation via interferometric synthetic aperture radar (InSAR), phase unwrapping procedures have difficulty in dealing with rough regions or large noise. Multiple baseline is used to reduce or avoid this problem. Conventional maximum likelihood (ML) methods reconstruct terrain heights in a pointwise fashion, which does not utilize the smooth characteristics of natural terrain. We propose a new algorithm taking smoothness into account. The new approach tackles the problem in a Bayesian framework. Instead of using ML estimation, we use maximum a posteriori (MAP) estimation, where the likelihood function is defined as in the ML method and the prior is defined as a first-order Gaussian Markov random field. This MAP estimation makes the algorithm more robust to noise, and at the same time, more accurate in reconstructing rough regions. A form of 2-D dynamic programming is used to implement the MAP estimation efficiently. The new algorithm has the advantage over the ML methods in that none of the baselines must be chosen so small as to avoid phase wrapping. Specifically, both baselines can be large so that the noise in the reconstructed height can be low. The new algorithm is shown to be able to achieve lower noise than the conventional ML and least-squares methods.