In an effort to increase the utilization of data center resources cloud providers have introduced a new type of virtual machine (VM) offering, called a burstable VM (BVM). Our work is the first to study the characteristics of burstable VMs (based on traces from production systems at a major cloud provider) and resource allocation approaches for BVM workloads. We propose new approaches for BVM resource allocation and use extensive simulations driven by field data to compare them with two baseline approaches used in practice. We find that traditional approaches based on using a fixed oversubscription ratio or based on the Central Limit Theorem do not work well for BVMs: They lead to either low utilization or high server capacity violation rates. Based on the lessons learned from our workload study, we develop a new approach to BVM scheduling, called Audible, using a non-parametric statistical model, which makes the approach light-weight and workload independent, and obviates the need for training machine learning models and for tuning their parameters. We show that Audible achieves high system utilization while being able to enforce stringent requirements on server capacity violations.

High density Solid State Drives, such as QLC drives, offer increased storage capacity, but a magnitude lower Program and Erase (P/E) cycles, limiting their endurance and hence usability. We present the design and implementation of non-binary, Voltage-Based Write-Once-Memory (WOM-v) Codes to improve the lifetime of QLC drives. First, we develop a FEMU based simulator test-bed to evaluate the gains of WOM-v codes on real world workloads. Second, we propose and implement two optimizations, an efficient garbage collection mechanism and an encoding optimization to drastically improve WOM-v code endurance without compromising performance. Third, we propose analytical approaches to obtain estimates of the endurance gains under WOM-v codes. We analyze the Greedy garbage collection technique with uniform page access distribution and the Least Recently Written (LRW) garbage collection technique with skewed page access distribution in the context of WOM-v codes. We find that although both approaches overestimate the number of required erase operations, the model based on greedy garbage collection with uniform page access distribution provides tighter bounds. A careful evaluation, including microbenchmarks and trace-driven evaluation, demonstrates that WOM-v codes can reduce Erase cycles for QLC drives by 4.4×–11.1× for real world workloads with minimal performance overheads resulting in improved QLC SSD lifetime.

This article presents the first large-scale field study of NAND-based SSDs in enterprise storage systems (in contrast to drives in distributed data center storage systems). The study is based on a very comprehensive set of field data, covering 1.6 million SSDs of a major storage vendor (NetApp). The drives comprise three different manufacturers, 18 different models, 12 different capacities, and all major flash technologies (SLC, cMLC, eMLC, 3D-TLC). The data allows us to study a large number of factors that were not studied in prior works, including the effect of firmware versions, the reliability of TLC NAND, and the correlations between drives within a RAID system. This article presents our analysis, along with a number of practical implications derived from it.

Regenerating codes are efficient methods for distributed storage in storage networks, where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrarily chosen storage nodes in the network. In this paper, we consider two simultaneous extensions to the original regenerating codes framework introduced by Dimakis et al.; 1) both data reconstruction and repair are resilient to the presence of a certain number of erroneous nodes in the network and 2) the number of helper nodes in every repair is not fixed, but is a flexible parameter that can be selected during the run-time. We study the fundamental limits of required total repair bandwidth and provide an upper bound for the storage capacity of these codes under these assumptions. We then focus on the minimum repair bandwidth (MBR) case and derive the exact storage capacity by presenting explicit coding schemes with exact repair, which achieve the upper bound of the storage capacity in the considered setup. To this end, we first provide a more natural extension of the well-known product matrix (PM) MBR codes, modified to provide flexibility in choosing the number of helpers in each repair, and simultaneously be robust to erroneous nodes in the network. This is achieved by proving the non-singularity for a family of matrices in large enough finite fields. We next provide another extension of the PM codes, based on a novel repair scheme which enables flexibility in the number of helpers and robustness against erroneous nodes without any extra cost in field size compared with the original PM codes.

In the distributed storage systems (DSSs) with k systematic nodes, robustness against node failure is commonly provided by storing redundancy in a number of other nodes and performing repair mechanism to reproduce the content of the failed nodes. Efficiency is then achieved by minimizing the storage overhead and the amount of data transmission required for data reconstruction and repair, provided by coding solutions, such as regenerating codes. Common explicit regenerating code constructions enable efficient repair through accessing a predefined number, d, of arbitrary chosen available nodes, namely helpers. In practice, however, the state of the system dynamically changes based on the request load, the link traffic, and so on, and the parameters which optimize system's performance vary accordingly. It is then desirable to have coding schemes which are able to operate optimally under a range of different parameters simultaneously. Specifically, adaptivity in the number of helper nodes for repair is of interest. While robustness requires capability of performing repair with small number of helpers, it is desirable to use as many helpers as available to reduce the transmission delay and total repair traffic. In this paper, we focus on the minimum storage regenerating (MSR) codes, where each of the n nodes in the network is supposed to store α information units, and the source data of size kα could be recovered from any arbitrary set of k nodes. We introduce a class of MSR codes that realize the optimal repair bandwidth simultaneously with a set of different choices for the number of helpers, namely D = {d1, . . . , dδ}. Our coding scheme follows the product matrix (PM) framework introduced by Rashmi et al. and could be considered as a generalization of the PM MSR code presented by Rashmi et al., such that any di= (i + 1)(k - 1) helpers can perform an optimal repair. As a result, the coding rate in our construction is limited by (k/n) ≤ (1/2). However, similar to the original design of PM MSR codes, our solution can realize practical values of the parameter α. Recently, Ye and Barg have presented another explicit MSR coding scheme which is capable of performing optimal repair for various number of helpers. The solution presented by Ye and Barg works for any arbitrary set of parameters k and D and can achieve high-coding rates, but the required α for this code is exponentially large. We show that the required value for α in the coding scheme presented in this paper is exponentially smaller when compared with the work of Ye and Barg for the same set of other parameters. Particularly, for a DSS with n nodes and k systematic nodes, the required value for α is reduced from sn to sk, where s = 1 cm(d1-k+1, ⋯, dδ-k+1). We also show the required field size in the presented coding scheme is equal to n.

Cloud computing is a promising framework providing a variety of solutions, ranging from software services to infrastructure services through the mechanism of customizable virtual instances. The cloud manager is responsible for resource provisioning for these instances to provide guaranteed performance but at the same time avoiding underutilization of the platform. In this paper, we introduce a novel method for modeling the resource usage of VIs which allows for better VI placement with more efficient resource usage in the physical infrastructure. Our proposed framework uses the mixture of Gaussians to model each virtual instance resource usage. Then for placement, a modified probabilistic bin packing method is been proposed to take advantage of modeling for placing virtual instances. We compared our scheme with other bin packing methods that use rigid statistical models, and the results support the efficiency and accuracy of our method which leads to more than 50% resource saving while preserving the given performance guarantee.

In coding for distributed storage systems, efficient data reconstruction and repair through accessing a predefined number of arbitrarily chosen storage nodes is guaranteed by regenerating codes. Traditionally, code parameters, specially the number of helper nodes participating in a repair process, are predetermined. However, depending on the state of the system and network traffic, it is desirable to adapt such parameters accordingly in order to minimize the cost of repair. In this work a class of regenerating codes with minimum storage is introduced that can simultaneously operate at the optimal repair bandwidth, for a wide range of exact repair mechanisms, based on different number of helper nodes.

Regenerating codes are efficient methods for distributed storage in practical networks where node failures are common. They guarantee low cost data reconstruction and repair through accessing only a predefined number of arbitrary chosen storage nodes in the network. In this work we study the fundamental limits of required total repair bandwidth and the storage capacity of these codes under the assumption that i) both data reconstruction and repair are resilient to the presence of a certain number of erroneous nodes in the network and ii) the number of helper nodes in every repair is not fixed, but is a flexible parameter that can be selected during the run-time. We focus on the minimum repair bandwidth point in this work, propose the associated coding scheme to posses both these extra properties, and prove its optimality.

We consider a streaming communication system where the source packets must be played back sequentially at the destination and study the associated average playback delay. We assume that all the source packets are available before the start of transmission at the transmitter and consider the case of an i.i.d. erasure channel with perfect feedback. We first consider the case when the receiver buffer can be arbitrarily large, and show that the average playback delay remains bounded in the length of the stream provided that the channel bandwidth is greater than a critical threshold. Our analysis involves the application of martingale theory to study the transient behaviour of a one dimensional random walk with drift. Conversely when the channel bandwidth is smaller than the above threshold, the average playback delay increases linearly with the stream length. We also consider the finite buffer case and analyse the playback delay of a greedy dynamic bandwidth scheme. We further show through simulations that the achievable delay with a finite receiver buffer is close to the infinite buffer case for moderately large buffer values.

We study a source-broadcasting problem involving an erasure broadcast channel with feedback. The receivers each require a certain fraction of a source sequence, and we are interested in the minimum latency, or transmission time, required to serve them all. We first show that for a two-user broadcast channel, a point-to-point outer bound can always be achieved. For broadcasting to three users, we propose a queue-based hybrid digital-analog coding scheme that achieves optimal performance for the duration of analog transmissions. We propose a method of characterizing the number of analog transmissions that can be sent, which involves solving a linear program, and furthermore give sufficient conditions for which all users can be optimal. In some cases, we find that users can be point-to-point optimal regardless of their distortion constraints. Finally, we propose a channel coding phase for when the analog transmissions are insufficient in meeting user demands and provide simulations that highlight the benefits of feedback.

Based on a new vision of the inactivation decoding process, we set a new degree distribution design criterion for the LT part of Raptor codes. Under an infinite block length assumption, a family of degree distributions that satisfy the new design criterion is analytically derived. The finite length performance of this family is investigated by using computer simulations and is shown to outperform the conventional design.

In this paper, an extension of raptor codes is introduced which keeps all the desirable properties of raptor codes, including the linear complexity of encoding and decoding per information bit, unchanged. The new design, however, improves the performance in terms of the reception rate. Our simulations show a 10% reduction in the needed overhead at the benchmark block length of 64,520 bits and with the same complexity per information bit.

A new receiver for convolutional coded information, which performs both the equalization and decoding in a joint scheme, based on the Viterbi algorithm is proposed. This method is based on creating a trellis for the joint system assuming some virtual auxiliary registers in the structure of the coder. In this way, the state space of the convolutional coder extends to support different states of the channel, enabling the receiver to include different states of the channel directly during the process of recovering the source information. Simulations indicate that the capability of taking the channel states into account during the decoding increases the performance of the overall receiver in the case of channels with memory. Also, the proposed method has lower complexity than a separated equalizer and hard decoder structure. A similar receiver for a memoryless channel can outperform conventional receivers. Simulation results for channels with and without memory are discussed.

In this paper a new method to resolve the overfitting problem for predicting complex systemspsila behavior has been proposed. This problem occurs when a neural network loses its generalization. The method is based on the training of recurrent neural networks and using simulated annealing for the optimization of their generalization. The major work is done based on the idea of ensemble neural networks. Finally the results of using this method on two sample datasets are presented and the effectiveness of this method is illustrated.