Integrating terrestrial networks together with non-terrestrial networks, such as Low Earth Orbit (LEO) satellites, can significantly enhance network reliability and reduce latency in unicast and multicast protocols. To use such a capability and reduce end-to-end latency, many transport-layer unicast protocols, such as Multipath TCP (MPTCP) and Multipath QUIC (MPQUIC), have adopted multi-connectivity (MC) by transmitting over multiple interfaces at the sender host. However, traditional and modern multicast protocols suppose that the sender host accesses the core network using a single path, leading to high end-to-end latency at destinations. In this regard, we first demonstrate in a real-world testbed that MC significantly improves network latency in multicasting compared to single-connectivity approaches. We then introduce a Bit Indexed Explicit Replication (BIER) Multi-Connectivity (BIER-MC) method designed to reduce end-to-end latency within the BIER protocol, as a modern multicast protocol. Our comparison indicates that BIER-MC outperforms traditional BIER implementations using multicast trees in terms of latency up to 5×, bandwidth usage up to 2.5×, and edge betweenness centrality up to 15×.

We address differentially private stochastic bandit problems by leveraging Thompson Sampling with Gaussian priors and Gaussian differential privacy (GDP). We propose DP-TS-UCB, a novel parametrized private algorithm that enables trading off privacy and regret. DP-TS-UCB satisfies $ \tilde{O} \left(T^{0.25(1-\alpha)}\right)$-GDP and achieves $O \left(K\ln^{\alpha+1}(T)/\Delta \right)$ regret bounds, where $K$ is the number of arms, $ \Delta$ is the sub-optimality gap, $T$ is the learning horizon, and $\alpha \in [0,1]$ controls the trade-off between privacy and regret. Theoretically, DP-TS-UCB relies on anti-concentration bounds for the Gaussian distributions, linking the exploration mechanisms of Thompson Sampling and Upper Confidence Bound, which may be of independent research interest.

Real-time communication (RTC) systems, such as video conferencing and cloud gaming, depend on effective congestion control (CC) algorithms to manage diverse network conditions and access technologies like Wi-Fi, LTE/5G, and satellite networks. While tools like AlphaRTC and Pandia have significantly advanced CC algorithm development for WebRTC, there is an absence of a unified framework for systematic benchmarking and cross-platform evaluation.

Video streaming performance may degrade substantially in a mobile environment due to fast-changing wireless links. On the other hand, to provide ubiquitous services, heterogeneous static and mobile access and backbone networks will be integrated in the sixth-generation (6G) systems, so mobile users can take advantage of multiple access options for better services. Multi-path transport-layer protocols like Multi-Path QUIC (MPQUIC) show promise in utilizing multiple access links to address the impact of mobility. However, the optimal link selection that aims to provide statistical QoS guarantee for video streaming in a mobile environment with both user mobility and network mobility remains an open issue. In this paper, based on a lightweight Multi-Armed Bandit (MAB) technique, we develop aQoS-drivenContextualMAB(QC-MAB) framework for MPQUIC, which makes an intelligent access network selection and adaptively enables FEC coding to trade off delay, reliability and goodput. Extensive simulation results with ns-3 show that the proposed QC-MAB framework can outperform the state-of-the-art solutions. It achieves up to ten times lower video interruption ratio and three times higher goodput in highly dynamic mobile environments.

In this paper, we study a multi-agent bandit problem in an unknown general-sum game repeated for a number of rounds (i.e., learning in a black-box game with bandit feedback), where a set of agents have no information about the underlying game structure and cannot observe each other’s actions and rewards. In each round, each agent needs to play an arm (i.e., action) from a (possibly different) arm set (i.e., action set), andonlyreceives the reward of theplayedarm that is affected by other agents’ actions. The objective of each agent is to minimize her own cumulative swap regret, where the swap regret is a generic performance measure for online learning algorithms. Many network optimization problems can be cast with the framework of this multi-agent bandit problem, such as wireless medium access control and end-to-end congestion control. We propose an online-mirror-descent-based algorithm and provide near-optimal high-probability swap-regret upper bounds based on refined martingale analyses, which can further bound the expected swap regret instead of the pseudo-regret studied in the literature. Moreover, the high-probability bounds guarantee that correlated equilibria can be achieved in a polynomial number of rounds if the algorithms are played by all agents. To assess the performance of the studied algorithm, we conducted numerical experiments in the context of wireless medium access control, and we performed emulation experiments by implementing the studied algorithms through the Linux Kernel for the end-to-end congestion control.

Heterogeneous deep learning clusters commonly host a variety of distributed learning jobs. In such scenarios, the training efficiency of learning models is negatively affected by the slowest worker. To accelerate the training process, multiple learning jobs may compete for limited computational resources, posing significant challenges to multi-job placement among heterogeneous workers. This paper presents a heterogeneity-aware scheduler to solve the multi-job placement problem while taking into account job sizing and load balancing, minimizing the average Job Completion Time (JCT) of deep learning jobs. A novel scheme based on proportional training workload assignment, feasible solution categorization, and matching markets is proposed with theoretical guarantees. To further reduce the computational complexity for low latency decision-making and improve scheduling fairness, we propose to construct the sparsification of feasible solution categories through sampling, which has negligible performance loss in JCT. We evaluate the performance of our design with real-world deep neural network benchmarks on heterogeneous computing clusters. Experimental results show that, compared to existing solutions, the proposed sampling-based scheme can achieve 1) results within 2.04% of the optimal JCT with orders-of-magnitude improvements in algorithm running time, and 2) high scheduling fairness among learning jobs.

The rapid growth of data processing demands in large-scale data centers (DCs) has led to increased brown energy (BE) consumption, which has negative environmental impacts. Since most DCs are now powered by both BE and renewable energy (RE), migrating workloads from DCs with insufficient RE to DCs with sufficient RE can decrease the total BE consumption in the network. However, selecting a destination DC is challenging due to the uncertainty of the network and the additional cost associated with using network devices for the migration. This paper proposes to optimize the DC selection and the efficient virtual machine transfer between DCs, minimizing the costs of BE consumption, optical network devices, and migration. Specifically, we formulate the DC selection as a multi-armed bandit problem and estimate the lowest migration cost at each round using the lower confidence bound. We adopt the optical grooming technique to reduce the cost of optical devices used during the migration. We compare our algorithm with the KUBE and -Greedy algorithms on the NSFNET and show that it reduces the total cost by 4.6% and 12.8%, respectively, while having lower regret. We demonstrated the effectiveness of optical grooming by achieving a 12 % reduction in network costs.

In this paper, we study the open problems raised by Karp et al. in FOCS 2000, where the authors formulated the end-to-end congestion control as a repeated game between a flow and an adversary. They mentioned several open problems including finding equilibria in a more realistic game model for the situation where the available bandwidth is a result of competition among multiple flows instead of being chosen by an adversary, and designing the randomized algorithm to deal with the dynamic change of network bandwidth. Although there have been many game-theoretic works for congestion control, to the best of our knowledge, the above two problems still remain unsolved over the past decades. We take a step further to address the above two problems by first modeling the end-to-end congestion control as a repeated unknown general-sum game among multiple flows with bandit feedback. Each flow is a player in this unknown game, making decisions on how many packets to send. The throughput for each flow depends on all the flows' rates and the network capacity. The unknown setting and bandit feedback capture the essence of end-to-end congestion control: each flow has no information about others (e.g., the number, actions, and packet loss of other flows), and only receives limited information for its chosen action. Then, we propose a randomized no-regret learning algorithm for each flow called LUC based on a swap-regret-minimizing technique. We prove that LUC can guarantee a polynomial-time convergence rate to correlated equilibria in the multi-player setting. Finally, we have implemented LUC through the Linux kernel, and conducted extensive fairness-related experiments in Mininet and trace-driven experiments with Pantheon to show that each flow with LUC can fairly share the bandwidth in homogeneous scenarios, and be competitive but TCP-friendly in heterogeneous scenarios.

In this paper, we study a multi-agent bandit problem in an unknown general-sum game repeated for a number of rounds (i.e., learning in a black-box game with bandit feedback), where a set of agents have no information about the underlying game structure and cannot observe each other’s actions and rewards. In each round, each agent needs to play an arm (i.e., action) from a (possibly different) arm set (i.e., action set), and only receives the reward of the played arm that is affected by other agents’ actions. The objective of each agent is to minimize her own cumulative swap regret, where the swap regret is a generic performance measure for online learning algorithms. We are the first to give a near-optimal high-probability swap-regret upper bound based on a refined martingale analysis for the exponential-weighting-based algorithms with the implicit exploration technique, which can further bound the expected swap regret instead of the pseudo-regret studied in the literature. It is also guaranteed that correlated equilibria can be achieved in a polynomial number of rounds if the algorithm is played by all agents. Furthermore, we conduct numerical experiments to verify the performance of the studied algorithm.

In this paper, we study a distributed stochastic multi-armed bandit problem that can address many real-world problems such as task assignment for multiple crowdsourcing platforms, traffic scheduling in wireless networks with multiple access points and caching at cellular network edge. We propose an efficient algorithm called multi-agent combinatorial upper confidence bound (MACUCB) with provable performance guarantees and low communication overhead. Furthermore, we perform extensive experiments to show the effectiveness of the proposed algorithm.

The burgeoning network traffic imposes a huge burden on the network backbone. Caching popular files at the wireless network edge is promising to address the problem. In practice, file popularity is very unlikely to know in advance. Online learning algorithms are effective to learn this uncertainty in a sequential way. In each slot, the learning agent generates a caching policy (i.e., the to-be-cached files) and can observe users' feedback about the caching policy within the same slot. This method implicitly requires that all of the users are able to provide feedback promptly. However, in practice, the availability of each individual user is affected by many factors, e.g., users are moving out of the service area temporarily, or they may still consume files in the previous slots, which may result in the feedback delay. In this paper, we propose a delay-tolerant wireless caching system that takes both the feedback delay and users' availability into consideration. We frame the content caching problem as a stochastic combinatorial multi-armed bandit problem with delayed feedback and forced-to-sleep arms, and devise an intelligent caching algorithm called CFAUD to solve the problem. Also, we show that CFAUD is effective and efficient both theoretically and practically. Finally, experiments are conducted to compare the performance of the proposed algorithm with other well-known algorithms.

Routing is a fundamental problem and has been extensively studied in various networks. However, in highly dynamic networks (e.g., wireless ad hoc networks), nodes have limited transmission opportunities due to high mobility, noise and interference, where traditional routing is often not the best approach.Opportunistic routing (OR), on the other hand, can effectively minimize the routing cost (e.g., the number of hops) and improve the success of routing by utilizing link metrics. However, the link metrics are usually unknown in advance and changing. In this paper, we design an adaptive algorithm calledThompson sampling-based opportunistic routing (TSOR)motivated by the distributed Bellman-Ford algorithms. TSOR is able to learn the link metrics and route packets simultaneously to reduce the overall cost. Theoretically, we show a lower bound and an upper bound of the cumulative regret (i.e., performance gap) between TSOR and the optimal routing algorithm that knows all link metrics in advance. The regret increases sublinearly with respect to the number of packets, and has a lower order in terms of the network size than the best-known results. Furthermore, we compare TSOR with the state-of-the-art algorithms, and the evaluation results show that TSOR has a lower regret and a faster convergence rate to the optimal policy than the state-of-the-art algorithms.

In recent years, with the rapid development and popularization of e-commerce, the applications of e-coupons have become a market trend. As a typical bar code technique, QR codes can be well adopted in e-coupon-based payment services. However, there are many security threats to QR codes, including the QR code tempering, forgery, privacy information leakage and so on. To address these security problems for real situations, in this paper, we introduce a novel fragment coding-based approach for QR codes using the idea of visual cryptography. Then, we propose a QR code scheme with high security by combining the fragment coding with the commitment technique. Finally, an enhanced QR code-based secure e-coupon transaction framework is presented, which has a triple-verification feature and supports both online and offline scenarios. The following properties are provided: high information confidentiality, difficult to tamper with and forge, and the ability to resist against collusion attacks. Furthermore, the performance evaluation of computing and communication overhead is given to show the efficiency of the proposed framework.

In recent years, with the rapid development and popularization of QR code-based services, the research of QR codes has become a hot topic. Because QR codes are easy to use, they are well adopted in e-coupon-based payment services. However, there are many security threats to QR codes, including QR code forgery, privacy information leakage and so on. To address these security problems, in this paper, we first propose a novel fragment coding-based approach for QR codes using the idea of visual cryptography. Second, we propose a QR code scheme with a high security by combining the fragment coding with commitment technique. Then, an enhanced QR code-based secure e-coupon transaction framework is presented, which has a triple verification feature. This framework can provide at least the following properties: high information confidentiality, difficult to tamper with and forge, and the ability to resist collusion attacks. Finally, security analysis and performance evaluation are presented to show the security and efficiency of the proposed framework.

A burgeoning number of wireless devices connecting to the Internet tend to impose a heavy traffic load on the network backbone. Caching the most popular content at the heterogeneous wireless network edge is a promising way to alleviate the network overload. However, to cache the diverse content effectively, a file popularity profile that may not be known in advance to network operators has to be utilized. To tackle the challenge caused by this uncertainty, online learning techniques can be considered. Additionally, in practice, dense small-cell networks are often deployed to maximize spectral efficiency, which will naturally bring overlapping coverage areas among individual small cells. In this paper, we propose to address the content caching problem in a scenario of overlapping coverage areas among small cells while further allowing users distributed in the overlapping area to stochastically choose to connect to the small-cell base station they can reach. We propose two effective and efficient online learning algorithms to address the aforementioned problem and also provide theoretical guarantees. Finally, experiments are conducted to verify the performance of the proposed algorithms practically.