Multiplicative Weight Update (MWU) is an online learning meta-algorithm that has been widely used in sequential decision-making problems. The key idea of MWU is to maintain a distribution as the selection probability allocation to the alternative actions. MWU has been independently rediscovered and investigated in many studies. This paper will revisit MWU from the perspective of fair resource allocation. Specifically, we focus on a well-known fairness concept called weighted alpha-fairness, which was introduced by Mo and Walrand in network bandwidth allocation. We first unveil that MWU inherently adopts the weighted alpha-fair probability allocation in each slot. The fairness level equals to the inverse of the step-size parameter in MWU, and the weight assigned to each action decays exponentially in relation to its accumulated cost. This insight suggests that the use of appropriate weighted-alpha fair allocations for action-taking probability can indeed perform as well as the offline optimal allocation in the long-term. Motivated by this insight, we further investigate whether one could solve more general online resource allocation problems (not limited to the one addressed by MWU) by carefully constructing the weighted-alpha fair allocation for each time slot. To this end, we propose a weighted alpha-fair online allocation framework that carefully constructs the merit-based weights and the increasing fairness levels. Under our proposed framework, the allocation outcome in each time slot satisfies the weighted alpha-fairness principal, and the allocation outcomes also perform asymptotically as well as the offline optimal outcome. We demonstrate how this framework functions in two networking applications. In size-based traffic scheduling, our framework enables the network switches to prioritize short flows and avoid flow starvation without the prior flow size information. In file caching, our framework outperforms several state-of-the-art caching policies up to 21% in terms of cache-hit-ratio.

OBJECTIVES: This study aims to (1) review machine learning (ML)-based models for early infection diagnostic and prognosis prediction in post-acute care (PAC) settings, (2) identify key risk predictors influencing infection-related outcomes, and (3) examine the quality and limitations of these models. MATERIALS AND METHODS: PubMed, Web of Science, Scopus, IEEE Xplore, CINAHL, and ACM digital library were searched in February 2024. Eligible studies leveraged PAC data to develop and evaluate ML models for infection-related risks. Data extraction followed the CHARMS checklist. Quality appraisal followed the PROBAST tool. Data synthesis was guided by the socio-ecological conceptual framework. RESULTS: Thirteen studies were included, mainly focusing on respiratory infections and nursing homes. Most used regression models with structured electronic health record data. Since 2020, there has been a shift toward advanced ML algorithms and multimodal data, biosensors, and clinical notes being significant sources of unstructured data. Despite these advances, there is insufficient evidence to support performance improvements over traditional models. Individual-level risk predictors, like impaired cognition, declined function, and tachycardia, were commonly used, while contextual-level predictors were barely utilized, consequently limiting model fairness. Major sources of bias included lack of external validation, inadequate model calibration, and insufficient consideration of data complexity. DISCUSSION AND CONCLUSION: Despite the growth of advanced modeling approaches in infection-related models in PAC settings, evidence supporting their superiority remains limited. Future research should leverage a socio-ecological lens for predictor selection and model construction, exploring optimal data modalities and ML model usage in PAC, while ensuring rigorous methodologies and fairness considerations.

Optimizing network protocols is crucial for improving application performance. Recent research works use multi-armed bandit (MAB) online learning methods to address network optimization problems, aiming to improve cumulative payoffs such as network throughput. However, existing MAB frameworks are ineffective since they inherently assume the network environment is static, or they have high complexity in detecting environmental changes. In this work, we advocate using lightweight "network-assist" techniques together with online learning to optimize network protocols, and show it can effectively detect environmental changes and maximize network performance. Furthermore, optimizing network protocols often face two types of decision (or arm) spaces: discrete and continuous choices, while most prior MAB models only handle discrete settings. This paper proposes a framework capable of managing both spaces. To our best knowledge, we are the first to develop an MAB framework that incorporates network-assist signals in handling dynamic environments, while considering the distinct characteristics of discrete and continuous arm spaces. Our framework can achieve optimality by showing its sub-linear regret bound, matching the state-of-the-art results in several degenerate cases. We also illustrate how to apply our framework to two network applications: (1) wireless network channel selection, and (2) rate-based TCP congestion control. We demonstrate the merits of our algorithms via both numerical simulations and packet-level experiments.

Multi-armed bandits (MAB) is an online learning and decisionmaking model under uncertainty. Instead of maximizing the expected utility (or reward) in a classical MAB setting, the variance of the utility should be considered when making risk-aware decisions. In this paper, we propose a risk-aware multi-agent MAB (MAMAB) model, which considers both the "independent" and "correlated" risk when multiple agents make arm-pulling decisions. Specifically, the system includes a platform that owns a number of tasks (or arms) awaiting a group of agents to accomplish. We show how to calculate the arm-pulling strategy of agents with potentially different eligible arm sets under a Nash equilibrium point. From the perspective of the platform, each arm has its maximal capacity to accommodate arm-pulling agents. We design the platform's optimal payment algorithms for its risk-aware revenue maximization (a regret minimization) under both independent and correlated risks. We prove that our algorithms achieve the sub-linear regret under independent risks when the platform can or cannot differentiate the utility on each arm. We also prove that our algorithm achieves the sublinear regret under correlated risks. We also carry out experiments to quantify the merits of our algorithms for various networking applications, such as crowdsourcing and edge computing.

Mobile edge computing facilitates users to offload computation tasks to edge servers for meeting their stringent delay requirements. Previous works mainly explore task offloading when system-side information is given (e.g., server processing speed, cellular data rate), or centralized offloading under system uncertainty. But both generally fall short of handling task placement involving many coexisting users in an uncertain environment. In this paper, we develop amulti-useroffloading framework consideringunknown yet stochasticsystem-side information to enable adecentralized user-initiatedservice placement under overlapping server coverage. Specifically, we formulate the dynamic task placement as an online multi-user multi-armed bandit process, and propose a decentralized epoch based offloading (DEBO) to optimize user rewards which are subjected under network delay. We consider both cases without and with neighboring edge feedback once users’ tasks are processed, where the latter incorporates system-side information sharing among edge servers for an enhanced task placement. For both cases, we show that DEBO can gradually deduce the optimal user-server assignment during dynamic offloading, thereby achieving aclose-to-optimalservice performance andtight$O(\log _{2}\!\!T)$regret. Moreover, we generalize DEBO to various common scenarios such as unknown reward gap, dynamic entering or leaving of clients, and fair reward distribution, while further exploring when users’ offloaded tasks requireheterogeneouscomputing resources. Particularly, we accomplish a sub-linear regret for each of these instances. Real measurements based evaluations corroborate the superiority of our offloading schemes over state-of-the-art approaches in optimizing delay-sensitive rewards.

Mobile edge computing (MEC) facilitates computation offloading to edge server and task processing via device-to-device (D2D) collaboration. Existing works mainly focus on centralized network-assisted offloading solutions, which are unscalable to collaborations among massive users. In this paper, we propose a joint framework of decentralized D2D collaboration and task offloading for MEC systems with large populations. Specifically, we utilize the power of two choices for D2D collaboration, which enables users to assist each other in a decentralized manner. Due to short-range D2D communication and user movements, we formulate a mean field model on a finite-degree and dynamic graph to analyze the collaboration state evolution. We derive the existence, uniqueness and convergence of the state stationary point to provide a tractable collaboration performance. Complementing this D2D collaboration, we further build a Stackelberg game to model users’ task offloading, where the provider, managing many servers, is the leader to determine service prices, while users are followers to make offloading decisions. By embedding Stackelberg game into Lyapunov optimization, we develop an online offloading and pricing scheme, which can optimize servers’ service utility or fairness, and users’ system cost simultaneously. Extensive evaluations show that D2D collaboration can mitigate users’ workloads by 73.8% and fair pricing can promote servers’ utility fairness by 15.87%.

Link speed in datacenter networks (DCNs) keeps growing rapidly, inducing an increasingly large portion of network flows to become short flows which can be finished within one round-trip time (RTT). This phenomenon makes many existing congestion control schemes ineffective because they iteratively adjust the sending rate based on the latest congestion feedback in multiple rounds. We find that the representative DCQCN scheme for RDMA exhibits substantial performance degradation when there are many short flows, and this is specially true in High Performance Computing (HPC) scenarios where most of Message Passing Interface (MPI) messages are small. In this paper, we propose a data-driven rate control framework which can learn from long-term online data about past rate control decisions via a lightweight online learning technique named Multi-Armed Bandit (MAB) which has a provable performance guarantee. Utilizing the framework, we devise a rate control scheme named Dolce-RC, which dynamically controls the rate increase and reduction by learning from online data. We implement Dolce-RC in commodity smart NICs, and show via testbed experiments and large-scale simulations that compared to DCQCN, Dolce-RC reduces average completion time of MPI messages by up to 68%, while not requiring any modification to switches.

Edge computing provides a platform facilitating edge servers to contribute to computation offloading while economizing their resources. Traditional offloading solutions are mostly centralized, which are unscalable for large-scale edge computing networks due to complex interactions among many edge servers. Meanwhile, dynamic pricing for an operator is equally, if not more, important to accommodate users’ time-varying demands for computing services. In this paper, we develop a decentralized online optimization framework to jointly minimize the server’s cost of workload scheduling while maximizing the operator’s utility of service pricing. Specifically, we employ the mean field game to model the collective scheduling behavior of all edge servers, thereby enabling optimal decision making only based on the server’s local information. Considering the service price in practice is not adjusted as frequently as the scheduling process, we establish a two-timescale optimization framework, where workload scheduling at a small timescale is tightly embedded into service pricing at a large timescale. Using mean field approximation, we derive the closed-form expression for the minimum scheduling cost, and the approximation error is$O\left ({\frac {1}{\sqrt {M}}}\right)$which declines as the number of edge servers$M$increases. By characterizing the influence of workload scheduling on dynamic pricing, we transform the complex service utility maximization into a succinct but equivalent problem, and thus we can make use of Lyapunov optimization to determine the optimal price over time. Extensive evaluations validate the effectiveness and optimality of our scheduling and pricing schemes.

Mobile edge computing facilitates users to offload computation tasks to edge servers for meeting their stringent delay requirements. Previous works mainly explore task offloading when system-side information is given (e.g., server processing speed, cellular data rate), or centralized offloading under system uncertainty. But both generally fall short of handling task placement involving many coexisting users in a dynamic and uncertain environment. In this paper, we develop a multi-user offloading framework considering unknown yet stochastic system-side information to enable a decentralized user-initiated service placement. Specifically, we formulate the dynamic task placement as an online multi-user multi-armed bandit process, and propose a decentralized epoch based offloading (DEBO) to optimize user rewards which are subject to the network delay. We show that DEBO can deduce the optimal user-server assignment, thereby achieving a close-to-optimal service performance and tight O(log T ) offloading regret. Moreover, we generalize DEBO to various common scenarios such as unknown reward gap, dynamic entering or leaving of clients, and fair reward distribution, while further exploring when users’ offloaded tasks require heterogeneous computing resources. Particularly, we accomplish a sub-linear regret for each of these instances. Real measurements based evaluations corroborate the superiority of our offloading schemes over state-of-the-art approaches in optimizing delay-sensitive rewards.

There is a growing trend that network applications not only require higher throughput, but also impose stricter delay requirements. The current Internet congestion control, which is driven by active queue management (AQM) algorithms interacting with the Transmission Control Protocol (TCP), has been playing an important role in supporting network applications. However, it still exhibits many open issues. Most of AQM algorithms only deploy a single-queue structure that cannot differentiate flows and easily leads to unfairness. Moreover, the parameter settings of AQM are often static, making them difficult to adapt to the dynamic network environments. In this paper, we propose a general framework for designing "self-tuning" queue management (SQM), which is adaptive to the changing environments and provides fair congestion control among flows. We first present a general architecture of SQM with fair queueing and propose a general fluid model to analyze it. To adapt to the stochastic environments, we formulate a stochastic network utility maximization (SNUM) problem, and utilize online convex optimization (OCO) and control theory to develop a distributed SQM algorithm which can self-tune different queue weights and control parameters. Numerical and packet-level simulation results show that our SQM algorithm significantly improves queueing delay and fairness among flows.

The classic utility maximization framework studies the fairness-efficiency tradeoff in various resource allocation problems (e.g., bandwidth allocation). The weighted alpha-fair utility is a common utilitarian metric. However, this classic framework cannot tackle those allocation problems with the online decision-making requirement (e.g., caching capacity allocation under unknown requests). Existing studies on these online allocation problems largely follow the online learning approaches, thus inevitably overlook the allocation fairness. In this paper, we propose a novel utility maximization framework accommodating the online setting. The major challenge of designing this framework lies in the tight coupling between the desirable fairness guarantee and the unknown allocation efficiency. To tackle this, we integrate the weighted alpha-fair utility with the learning rationale, by properly devising the merit-based weights and the increasing fairness levels. Under our proposed framework, the utility-maximizing allocation in each time slot is weighted alpha-fair. Our framework also performs asymptotically as well as the offline optimal/efficient outcome. We demonstrate how this framework functions in two networking applications. In size-based scheduling, it enables network switches to prioritize short flows and avoid flow starvation without the prior flow size information. In file caching, our framework outperforms several state-of-the-art caching policies up to 21% in terms of cache-hit-ratio.

The efficiency of a large-scale edge computing system primarily depends on three aspects: i) edge server provision, ii) task migration, and iii) computing resource configuration. In this paper, we study the dynamic resource configuration for hybrid edge server provision under two decentralized task migration schemes. We formulate the dynamic resource configuration as an online cost minimization problem, aiming to jointly minimize performance degradation and operation expenditure. Due to the stochastic nature, it is an online learning problem with partial feedback. To address it, we derive a deterministic mean field model to approximate the stochastic edge computing system. We show that the mean field model provides the increasingly accurate full feedback as the system scales. We then propose a learning policy based on the mean field model, and show that our proposed policy performs asymptotically as well as the offline optimal configuration. We provide two ways of setting the policy parameters, which achieve a constant competitive ratio (under certain mild conditions) and a sub-linear regret, respectively. Numerical results show that the mean field model significantly improves the convergence speed. Moreover, our proposed policy under the decentralized task migration schemes considerably reduces the operating cost (by 23%) and incurs little communication overhead.

The scheduling policy installed on switches of datacenters plays a significant role on congestion control. Shortest-Remaining-Processing-Time (SRPT) achieves the near-optimal average message completion time (MCT) in various scenarios, but is difficult to deploy as viewed by the industry. The reasons are two-fold: 1) many commodity switches only provide FIFO queues, and 2) the information of remaining message size is not available. Recently, the idea of emulating SRPT using only a few FIFO queues and the original message size has been coined as the approximate and deployable SRPT (ADS) design. In this paper, we provide the first theoretical study on the optimal ADS design. Specifically, we first characterize a wide range of feasible ADS scheduling policies via a unified framework, and then derive the steady-state MCT, slowdown, and impoliteness in the M/G/1 setting. Hence we formulate the optimal ADS design as a non-linear combinatorial optimization problem, which aims to minimize the average MCT given the available FIFO queues. We also take into account the proportional fairness and temporal fairness constraints based on the maximal slowdown and impoliteness, respectively. The optimal ADS design problem is NP-hard in general, and does not exhibit monotonicity or sub-modularity. We leverage its decomposable structure and devise an efficient algorithm to solve the optimal ADS policy. We carry out extensive flow-level simulations and packet-level experiments to evaluate the proposed optimal ADS design. Results show that the optimal ADS policy installed on eight FIFO queues is capable of emulating the true SRPT.

Mobile edge computing (MEC) facilitates computation offloading to edge server, as well as task processing via device-to-device (D2D) collaboration. Existing works mainly focus on centralized network-assisted offloading solutions, which are unscalable to scenarios involving collaboration among massive users. In this paper, we propose a joint framework of decentralized D2D collaboration and efficient task offloading for a large-population MEC system. Specifically, we utilize the power of two choices for D2D collaboration, which enables users to beneficially assist each other in a decentralized manner. Due to short-range D2D communication and user movements, we formulate a mean field model on a finite-degree and dynamic graph to analyze the state evolution of D2D collaboration. We derive the existence, uniqueness and convergence of the state stationary point so as to provide a tractable collaboration performance. Complementing this D2D collaboration, we further build a Stackelberg game to model users’ task offloading, where edge server is the leader to determine a service price, while users are followers to make offloading decisions. By embedding the Stackelberg game into Lyapunov optimization, we develop an online offloading and pricing scheme, which could optimize server’s service utility and users’ system cost simultaneously. Extensive evaluations show that our D2D collaboration can mitigate users’ workloads by 73.8% and task offloading can achieve high energy efficiency.

The scheduling policy installed on switches of datacenters plays a significant role on congestion control. Shortest-Remaining-Processing-Time (SRPT) achieves the near-optimal average message completion time (MCT) in various scenarios, but is difficult to deploy as viewed by the industry. The reasons are two-fold: 1) many commodity switches only provide FIFO queues, and 2) the information of remaining message size is not available. Recently, the idea of emulating SRPT using only a few FIFO queues and the original message size has been coined as the approximate and deployable SRPT (ADS) design. In this paper, we provide the first theoretical study on ADS design. Specifically, we first characterize a wide range of feasible ADS scheduling policies via a unified framework, and then derive the steady-state MCT and slowdown in the M/G/1 setting. We formulate the optimal ADS design as a non-linear combinatorial optimization problem, which aims to minimize the average MCT given the available FIFO queues. To prevent the starvation of long messages, we also take into account the fairness condition based on the steady-state slowdown. The optimal ADS design problem is NP-hard in general, and does not exhibit monotonicity or sub-modularity. We leverage its decomposable structure and devise an efficient algorithm to solve the optimal ADS policy. Numerical results based on the realistic heavy-tail message size distribution show that the optimal ADS policy installed on eight FIFO queues is capable of emulating the true SRPT in terms of MCT and slowdown.

The performance of an edge computing system primarily depends on the edge server provision mode, the task migration scheme, and the computing resource configuration. This paper studies how to perform dynamic resource configuration for hybrid edge server provision under two decentralized task migration schemes. We formulate the dynamic resource configuration as a multi-period online cost minimization problem, aiming to jointly minimize the performance degradation (i.e., execution latency) and the operation expenditure. Due to the stochastic nature, one can only observe the system performance for the currently installed configuration, which is also known as the partial feedback. To overcome this challenge, we derive a deterministic mean field model to approximate the large-scale stochastic edge computing system. We then propose an online mean field aided resource configuration policy, and show that the proposed policy performs asymptotically as good as the offline optimal configuration. Numerical results show that the mean field model can significantly improve the convergence speed in the online resource configuration problem. Moreover, our proposed policy under the two decentralized task migration schemes considerably reduces the operating cost (by 23%) and incurs little communication overhead.

Patient-generated health data (PGHD), such as patient-reported outcomes and mobile health data, have been increasingly used to improve health care delivery and outcomes. Integrating PGHD into electronic health records (EHRs) further expands the capacities to monitor patients' health status without requiring office visits or hospitalizations. By reviewing and discussing PGHD with patients remotely, clinicians could address the clinical issues efficiently outside of clinical settings. However, EHR-integrated PGHD may create a burden for clinicians, leading to burnout. This study aims to investigate how interactions with EHR-integrated PGHD may result in clinician burnout. We identify the potential contributing factors to clinician burnout using a modified FITT (Fit between Individuals, Task and Technology) framework. We found that technostress, time pressure, and workflow-related issues need to be addressed to accelerate the integration of PGHD into clinical care. The roles of artificial intelligence, algorithm-based clinical decision support, visualization format, human-computer interaction mechanism, workflow optimization, and financial reimbursement in reducing burnout are highlighted.

Bufferbloat is a phenomenon in computer networks where large router buffers are frequently filled up, resulting in high queueing delay and delay variation. More and more delay-sensitive applications on the Internet have made this phenomenon a pressing issue. Interacting with the Transmission Control Protocol (TCP), active queue management (AQM) algorithms run on routers play an important role in combating bufferbloat. However, AQM algorithms have not been widely deployed due to complicated manual parameter tuning. Moreover, they are often designed and analyzed based on network models with a single bottleneck link, rendering their performance and stability unclear in multi-bottleneck networks. In this paper, we propose a general framework to combat bufferbloat in multi-bottleneck networks. We first present an equilibrium analysis for a general multi-bottleneck TCP/AQM system and provide sufficient conditions for the uniqueness of an equilibrium point in the system. We then decompose the system into single-bottleneck subsystems and derive sufficient conditions for the local asymptotic stability of the subsystems. Using our framework, we develop an algorithm to compute the equilibrium point of the system. We further present a case study to analyze the stability of the recently proposed Controlled Delay (CoDel) in multi-bottleneck networks and devise Self-Tuning CoDel to improve the system stability. Extensive numerical and packet-level simulation results not only verify our theoretical studies but also show that our proposed Self-Tuning CoDel significantly stabilizes queueing delay in multi-bottleneck networks, thereby mitigating bufferbloat.

Bufferbloat is a phenomenon where router buffers are constantly being filled, resulting in high queueing delay and delay variation. Larger buffer size and more delay-sensitive applications on the Internet have made this phenomenon a pressing issue. Active queue management (AQM) algorithms, which play an important role in combating bufferbloat, have not been widely deployed due to complicated manual parameter tuning. Moreover, AQM algorithms are often designed and analyzed based on models with a single bottleneck link, rendering their performance and stability unclear in multi-bottleneck networks. In this paper, we propose a general framework to combat bufferbloat in multi-bottleneck networks. We first conduct an equilibrium analysis for a general multi-bottleneck TCP/ AQM system and develop an algorithm to compute the equilibrium point. We then decompose the system into single-bottleneck subsystems and derive sufficient conditions for the local asymptotic stability of the subsystems. Using the proposed framework, we present a case study to analyze the stability of the recently proposed Controlled Delay (CoDel) in multi-bottleneck networks and devise Self-tuning CoDel to improve the system stability and performance. Extensive simulation results show that Self-tuning CoDel effectively stabilizes queueing delay in multi-bottleneck scenarios, and thus contributes to combating bufferbloat.

Rate adaptation is extremely crucial to the system performance of wireless networks. Existing rate adaptation schemes mainly make use of channel information (e.g., packet error rate or signal strengths of received packets) to adapt transmission rates. In this paper, we find out that it is beneficial for rate adaptation schemes to consider priorities of packets when adapting transmission rates. We then propose a priority-based rate adaptation scheme for vehicular networks which jointly considers channel conditions and priorities of packets using game theory. In our scheme, we consider rate adaptation as a game which consists of different priorities of users and adopt a Stackelberg game model to regulate behaviors of self-interested users. Extensive ns-2 simulations demonstrate that the proposed scheme can provide much better performance for high priority users than existing schemes, while maintaining good performance for low priority users.