Current data centers rely on failure protection mechanisms to ensure data reliability. However, recent research indicates that failures within the same node or rack are common in data centers that use flash-based solid-state drives (SSDs) as the primary storage medium. Such correlated failures bring challenges for traditional protection mechanisms to achieve high reliability and repair performance. To this end, we propose a product erasure code (PECode) that encodes data blocks in multiple stripes cooperatively to generate intrastripe and interstripe parity blocks. Then, we design a multistripe cooperative repair algorithm (MSCRepair). MSCRepair first creates the failure distribution matrix (FDM) to represent the distribution of failure blocks in nodes and racks, and then conducts FDM-guided repair to minimize cross-rack traffic upon correlated failures. We prove that MSCRepair achieves the least cross-rack repair traffic at the cost of a longer repair time. We further propose a correlated failure repair scheduling algorithm for MSCRepair, which reduces the repair time by balancing the load and delivering data from links with higher bandwidths. We evaluate MSCRepair through both large-scale simulations and real experiments. In the mise-en-scene of its state-of-the-art alternatives, MSCRepair stands out by reducing up to 19.6%–49.9% of cross-rack traffic, while simultaneously reducing 16.2%–51.4% of recovery time of correlated failures.

Erasure coding is a storage-efficient redundancy scheme for modern key–value (KV) stores, storing stripes of data and parity chunks in multiple nodes. To accommodate the highly skewed and time-varying nature of the workload, KV stores require erasure code that dynamically optimizes its parameters, known as redundancy converting. Stretched Reed–Solomon (SRS) and elastic Reed–Solomon (ERS) codes represent promising candidates for meeting such requirements. However, both SRS and ERS are limited to RS$(d,r)\to $RS$(d^{\prime },r^{\prime })$converting, where$d^{\prime }>d,r^{\prime }=r$, failing to fully meet actual needs. This work presents an advanced ERS code (AERS code), which builds upon flexible encoding matrices and placement strategies, serving different types of redundancy converting, and minimizing converting traffic. We further prove that the AERS code is an optimal redundancy converting solution that achieves the theoretical lower bound on data traffic during redundancy converting while guaranteeing node-level fault tolerance. We evaluate the AERS code through both mathematical analysis and experiments. In the mise-en-scène of its state-of-the-art alternatives, AERS stands out by reducing network traffic up to 50%–85.7% while accelerating redundancy converting.

Hierarchical federated learning (HFL) has recently emerged as a more practical machine learning (ML) paradigm, which enables edge servers (ESs) in close proximity to conduct partial model aggregation. Despite its utility, local training and model aggregation incur considerable computation and communication time. client selection (CS) has proven effective for minimizing latency. However, CS faces the following challenges in hierarchical federated learning (HFL). First, the accessible clients, computation resources and network bandwidth are time-varying and unpredictable. Second, certain dynamics can only be observed after the decisions are made. Third, multiple ESs face different unknown clients, increasing the difficulty of selecting clients in an online manner. Finally, resource usage may be excessively violated during the training process. Existing HFL researches are insufficient to tackle these challenges. This work proposes a multi- ESs CS framework (MCS), which is based on multiarmed bandit (MAB) technique. MCS aims to reduce the cumulative computation and communication time, using two algorithms: 1) an online learning-based CS algorithm (OCA) makes the CS decisions for each ES, based on empirical learning results; and 2) a randomized rounding algorithm (RRA) converts fractional decisions obtained by OCA into binary solutions. Theoretically, MCS can enjoy the sublinear regret and violation compared to the optimal strategy. Practically, extensive experiments on real-world data sets demonstrate the empirical superiority of MCS over multiple state-of-the-art algorithms in minimizing cumulative latency.

Large-scale distributed machine learning (ML) systems involve extensive and costly computational resources. Pricing and scheduling, as two promising techniques for resource management, have garnered significant attention. However, existing job pricing and scheduling algorithms in cloud computing either charge fixed resource fees based on known job runtime or implement dynamic price setting with job preemption, unsuitable for distributed ML systems with high uncertainties and switching cost. First, whether the resources of a distributed ML job are placed together or not results in different job runtime. Second, various time-varying factors, including job arrival rates and competitors’ pricing, affect resource prices. Third, frequent price changes for the same resource can easily lead to system instability, ultimately jeopardizing user satisfaction. Addressing these uncertainties is challenging. This article introducesAPOS, an adaptive pricing and online scheduling framework, aiming at maximizing the operator’s overall revenue.APOSincorporates two innovations: 1) Intelligent Pricing: We represent each price using a feature vector that encapsulates relevant factors. Subsequently, based on the linear upper confidence bound (UCB) techniques, we establish relationships between price features and two revenue-associated elements: a) job arrival rates and b) resource consumption rates. To ensure system stability, we introduce batch pricing to reduce the frequency of resource price updates and 2) Online Scheduling: We strive to compute a nonpreemptive schedule that balances job utility with corresponding resource cost. We rigorously prove thatAPOSachieves truthfulness, individual rationality, system stability, and sublinear regret in polynomial time. Finally, extensive trace-driven simulations confirm thatAPOSoutperforms four state-of-the-art baselines, yielding a minimum of 23.3% improvement in total operator revenue.

As a new paradigm, edge caching is deemed an effective alternative by fetching contents at the network edge. However, designing an efficient caching mechanism is challenging. First, the content library is a dynamic set rather than a static set. Second, the content may be prevalent in different small base stations (SBSs), resulting in different rewards. Thus, the above reasons require each SBS could learn its caching decisions in a multi-SBSs network. Existing reinforcement learning algorithms either fail to consider the non-stationary environment or do not provide any performance guarantee. Thus, previous algorithms work well no longer. This work proposes a multi-agent multi-armed bandit caching framework, MAMAB-C, which navigates SBSs to cache contents in a distributed manner. Specifically, we formulate the multi-SBSs caching optimization problem as an online integer linear program (ILP) and convert it into a multi-agent multi-armed bandit (MAMAB) problem with resource constraints. MAMAB-C can realize the sub-linear metric property and significantly outperform multiple state-of-the-art algorithms.

By establishing a single parity check relationship between convolutional codewords, a concatenated code, termed single parity check convolutional code (SPC-CC), is proposed. By jointly en/decoding the SPC and CC, as well as carefully allocating redundant information between the pair, we can improve BER more promptly during iterative decoding. Each codeword in SPC-CC consists of only one SISO decoder, which generates one extrinsic information. The key is to simulate another using extrinsic information from other decoders. Then iteratively feed back extrinsic information to each other in a manner similar to a Turbo engine. We design en/decoding scheme of the SPC-CC, and then analyze its performance and complexity. Simulation results show that SPC-CC can effectively improve communication reliability with a moderate complexity. In addition, thanks to SPC code, SPC-CC can correct packet erasures due to fading or interference.