Though caching on edge servers is widely acknowledged to be essential, it is not trivial to cache content on edge servers adaptively without any prior knowledge of the distribution of content popularity across the users. Several edge caching algorithms have been proposed in the literature based on multi-agent reinforcement learning (MARL) for dynamic control, however, they ignored the non-stationarity and partial-observability issues commonly existing in multi-agent systems. In an MARL-based edge caching application where agents collaborate towards a common goal, communication is essential as their decisions are jointly applied to improve collective intelligence. However, most existing methods proposed to exchange messages between agents have not considered the induced communication overhead, which is critical in practice with real-world multi-agent applications. In this paper, we propose a new MARL framework for edge caching where agents learn to construct, exchange and interpret collective messages for individual benefits, while controlling the complex collaborative task of cache replacement in a communication-efficient manner. With a standard edge caching model, we show that with limited communication and delays introduced, our proposed framework is able to outperform existing rule-based and learning-based caching policy alternatives.

We propose SutraNets, a novel method for neural probabilistic forecasting of long-sequence time series. SutraNets use an autoregressive generative model to factorize the likelihood of long sequences into products of conditional probabilities. When generating long sequences, most autoregressive approaches suffer from harmful error accumulation, as well as challenges in modeling long-distance dependencies. SutraNets treat long, univariate prediction as multivariate prediction over lower-frequency sub-series. Autoregression proceeds across time and across sub-series in order to ensure coherent multivariate (and, hence, high-frequency univariate) outputs. Since sub-series can be generated using fewer steps, SutraNets effectively reduce error accumulation and signal path distances. We find SutraNets to significantly improve forecasting accuracy over competitive alternatives on six real-world datasets, including when we vary the number of sub-series and scale up the depth and width of the underlying sequence models.

Federated learning preserves data privacy by training machine learning models in a distributed fashion, where local models are trained on the client devices and aggregated on the server. Prevalent aggregation algorithms in federated learning perform well in homogeneous settings, but suffer from inadequate convergence in heterogeneous settings due to non-IID data distribution. In this paper, we explore the shortcomings of existing work and recognize that the memory loss of optimizers in aggregation steps limits convergence performance. In response, we propose FedRL, a new adaptive aggregation algorithm with the supervision of a policy-based deep reinforcement learning agent. Using real-world datasets, we evaluate the effectiveness of FedRL by comparing to state-of-the-art adaptive aggregation algorithms in the literature, and show its superiority in accelerating convergence to a target accuracy.

We present coarse-to-fine autoregressive networks (C2FAR), a method for modeling the probability distribution of univariate, numeric random variables. C2FAR generates a hierarchical, coarse-to-fine discretization of a variable autoregressively; progressively finer intervals of support are generated from a sequence of binned distributions, where each distribution is conditioned on previously-generated coarser intervals. Unlike prior (flat) binned distributions, C2FAR can represent values with exponentially higher precision, for only a linear increase in complexity. We use C2FAR for probabilistic forecasting via a recurrent neural network, thus modeling time series autoregressively in both space and time. C2FAR is the first method to simultaneously handle discrete and continuous series of arbitrary scale and distribution shape. This flexibility enables a variety of time series use cases, including anomaly detection, interpolation, and compression. C2FAR achieves improvements over the state-of-the-art on several benchmark forecasting datasets.

Decision-making in large-scale compute clouds relies on accurate workload modeling. Unfortunately, prior models have proven insufficient in capturing the complex correlations in real cloud workloads. We introduce the first model of large-scale cloud workloads that captures long-range inter-job correlations in arrival rates, resource requirements, and lifetimes. Our approach models workload as a three-stage generative process, with separate models for: (1) the number of batch arrivals over time, (2) the sequence of requested resources, and (3) the sequence of lifetimes. Our lifetime model is a novel extension of recent work in neural survival prediction. It represents and exploits inter-job correlations using a recurrent neural network. We validate our approach by showing it is able to accurately generate the production virtual machine workload of two real-world cloud providers.

Scheduling complex workflows for big data systems is both fundamentally challenging and of great practical importance. Some state-of-the-art schedulers ignore important real-world considerations for the sake of algorithmic tractability, while others are tailored for specific workloads. We consider the preemption version of the Minimum Makespan Workflow Scheduling of Malleable Jobs with Precedence Constraints (MMWS-MP) problem [24], [27] and generalize it to be applicable to a broader range of real-world big data applications. In particular, we formulate MMWSMPL by introducing an additional constraint on the lifetime resource demand, which models constant resource consumption throughout the lifetime of a set of jobs. Practical examples include ApplicationMasters in YARN, port reservations, software licenses, and GPU cycles. We devise two scheduling strategies for MMWS-MPL: (1) LPSched, which takes a linear programming approach, and (2) BoltSched, a mostly greedy heuristic. We prove that LPSched achieves a constant approximation ratio of (2+ε) for any ε>0 and thus serves as a theoretically sound comparison baseline. We empirically evaluate both strategies on synthetic benchmarks and show that BoltSched produces schedules that are nearly as good as LPSched at a fraction (around 10%) of the computational cost.