Multi-tenant storage management environments typically manage multiple enterprise accounts with heterogeneous storage footprints. Identifying and grouping accounts with similar storage footprints into clusters reduces account management overhead, and provides a framework for data-driven storage recommendation services. This paper describes a method for the clustering of accounts in multi-tenant storage management environments. Storage system vendors, models, and footprints are captured as a set of properties, and a pairwise distance function is employed to grade similarity between accounts along multiple dimensions. A graph representation is created based on account similarity values. Finally, the clustering algorithm is defined as a graph algorithm with the purpose of repeatedly finding maximum cliques, and removing them from the graph. The result of the graph algorithm is a set of clusters, each grouping together accounts with very similar storage footprints. Clusters are then rated along multiple metrics, are compared to their peers along multiple performance dimensions, and receive recommendations on how to further improve storage efficiency.

With the advent of social media, Internet of Things (IoT), widespread use of richer media formats such as video, and generally increased use of mobile devices, volume of online data has seen a rapid increase in recent years. To cope with this data explosion, businesses and cloud providers are scrambling to lower the cost of storing data without sacrificing the quality of their service using space reduction techniques such as compression and deduplication. Capacity savings, however, are achieved at the cost of performance and additional resource overheads. One drawback of compression techniques is the additional computation required to store and fetch data, which may significantly increase response time, i.e., I/O latency. Worse yet, inefficient compression algorithms that fail to compress data satisfactorily suffer from the latency penalty with marginal capacity savings, e.g., deciding to compress data that is encrypted or already compressed. Therefore, from a data center administrator's perspective, we should pick the set of volumes that will yield the most compression space saving with the least latency for a given amount of computation capacity, without exhaustively inspecting the data content of volumes. To fill this void, this paper proposes an approach to manage compression for a very large set of volumes. It maximizes capacity savings and minimizes latency impact without scanning the actual data content (to avoid security concerns). Our pilot deployments show significant capacity savings and performance improvements compared to benchmark compression strategies.

Spectrum trading benefits primary users (PUs) by monetary gains and secondary users (SUs) by spectrum accessing opportunities in cognitive radio networks (CRNs). Unfortunately, most existing spectrum trading designs only focus on the guarantee of economic properties, but forget the wireless transmission nature, especially for multi-hop cognitive radio (CR) communications. In this paper, we propose an economic-robust session based spectrum trading, which has a joint consideration of economic properties such as incentive compatibility, individual rationality, and budget balance, and the end-to-end performance for multi-hop communications. Considering two bidding manners, i.e., bidding for the whole session and unit rate bidding, we formulate the spectrum trading optimization problems under multiple economic and multi-hop CR transmission constraints, design two pricing mechanisms to charge the winning spectrum bidders, and further mathematically prove the economic- robustness of the proposed spectrum trading schemes. Through extensive simulations, we show the proposed schemes are economic-robust and effective in improving spectrum utilization.

Frequent Item set Mining (FIM) is a classic data mining topic with many real world applications such as market basket analysis. Many algorithms including Apriori, FP-Growth, and Eclat were proposed in the FIM field. As the dataset size grows, researchers have proposed MapReduce version of FIM algorithms to meet the big data challenge. This paper proposes new improvements to the MapReduce implementation of FIM algorithm by introducing a cache layer and a selective online analyzer. We have evaluated the effectiveness and efficiency of Smart Cache via extensive experiments on four public datasets. Smart Cache can reduce on average 45.4%, and up to 97.0% of the total execution time compared with the state-of-the-art solution.

Storage management aims to improve the data center performance by optimizing the underlying storage resources more efficiently. The advent of cloud computing technologies introduces a paradigm shift from conventional on-premise storage management solutions to multitenant storage management as a service models. The advantage of centralized multitenant storage management platforms lies in the integrated procedures and the unified platform of data collection, governance, and analytics to gauge the effectiveness and efficiency of tenant storage environments. In this paper, we introduce our data-driven storage optimization framework, which is a centralized storage data analytics module to provide recommendations for the storage administrator. We use three example use cases to illustrate the exploitation of individual data center operational data to obtain actionable insights for better storage management solutions.

Continuous Data Protection (CDP) enables recoverability to any point in time (time travel) facilitated via journaling of every write made by a system to disk. Stringent storage performance and capacity requirements for journaling make CDP a very high cost solution leading to limited adoption. In this work we first explore the feasibility of building such a CDP function on top of cheap commodity storage exposed via cloud object stores. Based on this analysis, we propose cCDP - a Cloud Continuous Data Protection framework that efficiently combines cloud object stores with edge caching to address requirements of low cost, high capacity, low latency and high storage throughput. cCDP with careful tuning can not only meet but surpasses the write throughput and latency requirements of CDP with minimal buffer overheads (<; 1%). Recovery lookup performance (Identification & Ordering) of cCDP with hybrid data layout is within 28% of Btree based temporal data layout and is about 52.7% better than the naive data layout with name encoding with significantly lower edge buffer and write throughput overheads than the Btree based temporal data layout.

The capacity region of wireless networks with per-destination PD queueing model has been studied extensively in the literature. However, the PD queueing structure is not scalable because the number of queues in a node can be as large as the number of all possible source-destination pairs. In this work, we study the capacity region of wireless networks with per-link PL queueing model. The advantage of the PL queueing structure is that the number of queues in a node can be reduced significantly to the number of its neighboring nodes. In this paper, the capacity region of a wireless network with PL queueing structure is characterized, and a dynamic routing and power control policy, namely, DRPC-PL, is proposed to stabilize the network whenever the input rate is within the capacity region. Copyright © 2013 John Wiley & Sons, Ltd.

In enterprise data centers, reliable performance models on storage devices are desirable for efficient storage management and optimization. However, many cloud environments consist of heterogeneous storage devices, e.g., a mixture of commodity disks, where accurate performance models are of particular challenge to attain. In this paper, we propose a lightweight queueing-based storage performance modeling framework, which is able to infer the maximum IO load that a storage device can sustain, as well as its IO load v.s. response time performance curve. Our inference framework views the underlying storage resources as blackboxes and only utilizes historical measurements of the IO and response time on the devices. In an OpenStack environment, we also develop a new storage volume placement algorithm using our performance inference and modeling framework. Experimental results show that our solution can provide up to 80% increase of the IO throughput, in tandem with a 40% reduction of the average response time, compared to the performance provided by the default OpenStack policy.

Virtualization is a double-edged sword in large scale cloud computing environments. On one hand, virtualization provides a logical and unified view of the underlying cloud resources to facilitate more efficient resource utilization and to support multi-tenancy. On the other hand, virtualization introduces additional layers of indirections which make the virtual-to-physical resource mapping relationship obscure, or prohibitively costly to unveil, especially with multiple layers of virtualization at presence, e.g., server virtualization, network virtualization, and storage virtualization. In this work, we propose a virtual-to-physical mapping inference framework by analyzing the I/O activities of inputs and outputs of the virtualization layer. Using a virtualized storage example, we show that our lightweight solution can unveil the virtual-to-physical resource mapping relationship in a non-intrusive fashion, which can be used in many cloud management tasks such as root cause analysis and diagnosis.

Data protection using backup is one of the most critical IT management operations to ensure business continuity, which is also constantly evolving due to the emerging challenges in the “Big Data Era.” In this paper, we introduce our ongoing research effort in designing intelligent enterprise backup management solutions by obtaining actionable insights from voluminous backup job metadata across data centers. Our proof-of-concept scalable data analytics framework integrated with enterprise backup architecture is introduced, and use case studies of leveraging big data analytics on enterprise backup framework are discussed.

Spectrum trading creates more accessing opportunities for secondary users (SUs) and economically benefits the primary users (PUs). However, it is challenging to implement spectrum trading in multi-hop cognitive radio networks (CRNs) due to harsh cognitive radio (CR) requirements on SUs' devices, uncertain spectrum supply from PUs and complex competition relationship among different CR sessions. Unlike the per-user based spectrum trading designs in previous studies, in this paper, we propose a novel session based spectrum trading system, spectrum clouds, in multi-hop CRNs. In spectrum clouds, we introduce a new service provider, secondary service provider (SSP), to facilitate the accessing of SUs without CR capability and harvest uncertain spectrum supply. The SSP also conducts spectrum trading among CR sessions w.r.t. their conflicts and competitions. Leveraging a 3-dimensional (3-D) conflict graph, we mathematically describe the conflicts and competitions among the candidate sessions for spectrum trading. Given the rate requirements and bidding values of candidate trading sessions, we formulate the optimal spectrum trading into the SSP's revenue maximization problem under multiple cross-layer constraints. In view of the NP-hardness of the problem, we develop heuristic algorithms to pursue feasible solutions. Through extensive simulations, we show that the solutions found by the proposed algorithms are close to the optimal one.

Rate adaptation is a MAC-layer mechanism in IEEE 802.11 networks that ensures efficient utilization of constantly varying wireless channel. For broadcast communication, however, the current practice is to send data at base rates (i.e., 1 or 2 Mbps in 802.11b/b+g and 6Mbps in 802.11a/g), leaving the channel capacity significantly underutilized. In this paper, we present a practical rate adaptation framework designed for broadcasting of streaming data to improve the channel utilization of 802.11 networks. The proposed framework jointly selects the data rate and the error protection level of FEC in a novel manner, and does not require changes to either existent applications or to the 802.11 standards - thus can be deployed in existing 802.11 networks. Using the performance measurement study in various live networking environments, we show that the proposed scheme provides substantial improvement in wireless channel utilization (more than 10x improvement compared to the current broadcast mode) and can adapt to different operational environments.

Spot virtual-machine (VM) instances, such as Amazon EC2 Spot VMs, are a class of VMs that are purchased through a market mechanism of price-bids submitted by cloud users. Spot VMs can be obtained at substantially lower cost than other VM classes such as Reserved and On-demand instances, but they do not have guaranteed availability since it depends on the submitted price bids and the fluctuating spot VM price. Many applications with large computing requirements but no real-time availability constraints, such as scientific computing, financial modelling and large data analysis, can be carried out at a significantly lower cost using spot VMs. For such jobs, an important question that arises is what should the submitted price bids be so that the computation is completed within a fixed time interval while the cost is minimized. Towards this goal, we model a job as a fixed computation request with a deadline constraint and formulate the problem of designing a dynamic bidding policy that minimizes the average cost of job completion. We obtain analytical and closed-form results for the optimal strategy under a Markov spot price evolution, and then evaluate the performance of the algorithms on the actual spot price history of Amazon EC2 Spot VMs.

Spectrum trading creates more accessing opportunities for secondary users (SUs) and economically benefits the primary users (PUs). However, it is challenging to implement spectrum trading in multi-hop cognitive radio networks (CRNs) due to harsh cognitive radio (CR) requirements on SUs' devices and complex conflict and competition relationship among different CR sessions. Unlike the per-user based spectrum trading designs in previous studies, in this paper, we propose a novel session based spectrum trading system, spectrum clouds, in multi-hop CRNs. In spectrum clouds, we introduce a new service provider, called secondary service provider (SSP), to harvest the available spectrum bands and facilitate the accessing of SUs without CR capability. The SSP also conducts spectrum trading among CR sessions w.r.t. their conflicts and competitions. Leveraging a 3-dimensional (3-D) conflict graph, we mathematically describe the conflicts and competitions among the candidate sessions for spectrum trading. Given the rate requirements and bidding values of candidate trading sessions, we formulate the optimal spectrum trading into the SSP's revenue maximization problem under multiple cross-layer constraints in multi-hop CRNs. In view of the NP-hardness of the problem, we have also developed heuristic algorithms to pursue feasible solutions. Through extensive simulations, we show that the solutions found by the proposed algorithms are close to the optimal one.

Amazon introduced Spot Instance Market to utilize the idle resources of Amazon Elastic Compute Cloud (EC2) more efficiently. The price of a spot instance changes dynamically according to the current supply and demand for cloud resources. Users can bid for a spot instance and the job request will be granted if the current spot price falls below the bid, whereas the job will be terminated if the spot price exceeds the bid. In this paper, we investigate the problem of designing a bidding strategy from a cloud service broker's perspective, where the cloud service broker accepts job requests from cloud users, and leverages the opportunistic yet less expensive spot instances for computation in order to maximize its own profit. In this context, we propose a profit aware dynamic bidding (PADB) algorithm, which observes the current spot price and selects the bid adaptively to maximize the time average profit of the cloud service broker. We show that our bidding strategy achieves a near-optimal solution, i.e., (1-∈) of the optimal solution to the profit maximization problem, where ∈ can be arbitrarily small. The proposed dynamic bidding algorithm is self-adaptive and requires no a priori statistical knowledge on the distribution of random job sizes from cloud users.

In this paper, we study a wireless multi-hop network where multiple flows co-exist and share the network resource collectively. Each flow is associated with a user which has specific requirements on its tradeoff between cost and quality of service. To support heterogeneous transmissions efficiently, we propose a quality-aware dynamic pricing algorithm, namely, QADP, which provably maximizes the overall network revenue while maintaining the stability of the network. Our proposed scheme enjoys the merit of self-adaptability due to its online nature.

For a multi-hop wireless network (MWN) consisting of mobile nodes controlled by independent self-interested users, incentive mechanism is essential for motivating mobile nodes to cooperate and forward packets for each other. Existing solutions such as barter based, virtual-currency based and reputation based schemes are either less effective or incur high implementation costs, and therefore do not fit well with the unique requirements of MWNs. In this paper, we propose a novel and promising incentive paradigm, Controlled Coded packets as virtual Commodity Currency (C4), to induce cooperative behaviors in MWNs. In our C4, through introducing several techniques from network coding, coded information packets are utilized as a new kind of virtual currency to facilitate packet/service exchanges among self-interested nodes in a MWN. Since the virtual currency implemented in this way also carries useful data information, it is the counterpart of the so-called commodity currency in the physical world, and the overhead brought by C4 is extremely small compared to traditional schemes. We theoretically show that C4 is perfectly efficient to support MWNs with broadcast and multicast traffics. For pure unicast communications, by adjusting the grouping parameter, our C4 provides a systematic way to smoothly trade incentive effectiveness for implementation cost, and traditional barter based and virtual-currency based schemes are just two extreme cases of C4. We also show that when our C4 is combined with the social network formed by mobile users in the MWN, the implementation costs can be further reduced without sacrificing incentive effectiveness.

We consider a coalition network where multiple groups are interconnected via wireless links. Gateway nodes are designated by each domain to achieve a network-wide interoperability. Due to the inter-domain communication cost, the optimal gateway selection for one single domain depends on the gateway selections of other domains and vice versa. In this paper, we investigate the interactions of gateway selections by multiple domains from a potential game perspective. The equilibrium inefficiency in terms of price of stability is characterized under various conditions. In addition, we examine the well-established equilibrium selective learning algorithm B-logit and show that B-logit is a special case of a general family of algorithms, denoted by Γ collectively. A novel learning algorithm named MAX-logit is proposed, which retains the favorable equilibrium selection property with the provably fastest convergence speed than any other algorithms in Γ, and can be applied to many other applications of potential games. Simulation results show that MAX-logit can improve the convergence speed of B-logit by up to 33.85%.

Security always comes with a price in terms of performance degradation, which should be carefully quantified. This is especially the case for wireless ad hoc networks (WANETs), which offer communications over a shared wireless channel without any preexisting infrastructure. Forming end-to-end secure paths in such WANETs is more challenging than in conventional networks due to the lack of central authorities, and its impact on network performance is largely untouched in the literature. In this paper, based on a general random network model, the asymptotic behaviors of secure throughput and delay with the common transmission range rnand the probability pfof neighboring nodes having a primary security association are quantified when the network size n is sufficiently large. The costs and benefits of secure-link-augmentation operations on the secure throughput and delay are also analyzed. In general, security has a cost: Since we require all the communications operate on secure links, there is a degradation in the network performance when pffis Ω(1/logn), the secure throughput remains at the Gupta and Kumar bound of Θ(1/√{n log n}) packets/time slot, wherein no security requirements are enforced on WANETs. This implies that even when the pfgoes to zero as the network size becomes arbitrarily large, it is still possible to build throughput-order-optimal secure WANETs, which is of practical interest since pfis very small in many practical large-scale WANETs.

Cognitive Radio technology releases the spectrum from shackles of authorized licenses and facilitates the trading of spectrum bands. In the spectrum market, primary service providers (PSPs) set price for the vacant licensed bands of primary users (PUs) and sell them for monetary gains, and the secondary service provider (SSP) can buy the bands and opportunistically use them to satisfy the service demands of secondary users (SUs) when the primary services are not active. However, when there are multiple bands available, the SSP confronts the challenges of how to choose bands and how to split the overall traffic on them considering both his monetary reward and the potential risk, from the unpredictable activities of the primary services, for opportunistic spectrum accessing (OSA). In this paper, we propose a reward and risk based band-mix selection algorithm to address these concerns of the SSP, and help the SSP to make appropriate decisions of traffic splitting over available spectrum band- mix, consisting of both the band belonging to SSP itself and the bands from PSPs. By numerical simulations, we verify our theoretical analysis and show that the proposed spectrum band-mix selection effectively improves the spectrum utilization as well as the satisfactory degree of SUs.

MaxWeight algorithm, a.k.a., back-pressure algorithm, has received much attention as a viable solution for dynamic link scheduling in multi-hop wireless networks. The basic principle of the MaxWeight algorithm is to select a set of interference-free links with the maximum overall link weights in the network, where the link weight is determined by the queue difference between the transmitter and the receiver. While the throughput-optimality of the MaxWeight algorithm is well understood in the literature, the energy consumption induced by the MaxWeight algorithm is less studied, which is of great interest in energy-constrained wireless networks such as wireless sensor networks. In this paper, we propose an energy-conserving scheduling scheme, a.k.a., minimum energy scheduling (MES) algorithm for multi-hop wireless networks with stochastic traffic arrivals and time-varying channel conditions. We show that our algorithm is energy optimal in the sense that the proposed MES algorithm can achieve an energy consumption which is arbitrarily close to the global minimum solution. Moreover, the energy efficiency of the MES algorithm is achieved without losing the throughput- optimality. In other words, the proposed MES algorithm is still throughput optimal whereas the average consumed energy in the network is significantly reduced, as compared to the traditional MaxWeight algorithm. The theoretical results are substantiated via simulations.

Recently, trust-based routing has received much attention as an effective way to improve security of wireless ad hoc networks (WANETs). Although various trust metrics have been designed and incorporated into the routing metrics, as far as we know, none of the existing works have used mathematical tools such as routing algebra to analyze the compatibility of trust related routing metrics and routing protocols in WANETs. In this paper, we first identify unique features of trust metrics compared with QoS-based routing metrics. Then, we provide a systematic analysis of the relationship between trust metrics and trust-based routing protocols by identifying the basic algebraic properties that a trust metric must have in order to work correctly and optimally with different generalized distance-vector or link-state routing protocols in WANETs. Moreover, we extend our framework to model the interactions between different trust-based routing protocols. Finally, our results are applied to check the compatibility of the trust metrics proposed in previous literature and the popular routing protocols used in WANETs.

In this work, the stochastic traffic engineering problem in multihop cognitive wireless mesh networks is addressed. The challenges induced by the random behaviors of the primary users are investigated in a stochastic network utility maximization framework. For the convex stochastic traffic engineering problem, we propose a fully distributed algorithmic solution which provably converges to the global optimum with probability one. We next extend our framework to the cognitive wireless mesh networks with nonconvex utility functions, where a decentralized algorithmic solution, based on learning automata techniques, is proposed. We show that the decentralized solution converges to the global optimum solution asymptotically.

MaxWeight algorithm, a.k.a., back-pressure algorithm [1]-[4], has received much attention as a viable solution for dynamic link scheduling in multi-hop wireless networks. The basic principle of the MaxWeight algorithm is to select a set of interference-free links with the maximum overall link weights in the network, where the link weight is determined by the queue difference between the transmitter and the receiver. While the throughput-optimality of the MaxWeight algorithm is well understood in the literature, the energy consumption induced by the MaxWeight algorithm is less studied, which is of great interest in energy-constrained wireless networks such as wireless sensor networks. In this paper, we propose a minimum energy scheduling (MES) algorithm for multi-hop wireless networks with stochastic traffic arrivals and time-varying channel conditions. We show that our algorithm is energy optimal in the sense that the proposed MES algorithm can achieve an energy consumption which is arbitrarily close to the global minimum solution. Moreover, the energy efficiency of the MES algorithm is achieved without losing the throughput-optimality. In other words, the proposed MES algorithm is still throughput optimal whereas the average consumed energy in the network is significantly reduced, as compared to the traditional MaxWeight algorithm. The theoretical results are substantiated via simulations.

Rate adaptation algorithms play a crucial role in IEEE 802.11 WLANs. While the network performance depends greatly on the rate adaptation algorithms, the detailed implementation is left to vendors. Due to its simplicity and practicality, threshold-based rate adaptation algorithms are widely adopted in commercial IEEE 802.11 devices. Taking the popular ARF algorithm for example, the data rate is increased when ten consecutive transmissions are successful and a date rate downshift is triggered by two consecutive failed transmissions. Although widely deployed, the optimal selection of the up/down thresholds for the rate adaptation algorithms remains an open problem. In this paper, we first investigate the threshold-based rate adaptation algorithm via a reverse engineering approach where the implicit objective function is revealed. Next, we propose a threshold optimization algorithm which can dynamically adjust the up/down thresholds and converge to the stochastic optimum solution in arbitrary stationary random channel environment. The performance enhancement by tuning the thresholds optimally is validated by simulations.

Rate adaptation algorithms play a crucial role in IEEE 802.11 WLANs. While the network performance depends greatly on the rate adaptation algorithms, the detailed implementation is left to vendors. Due to its simplicity and practicality, the generic rate adaptation algorithm based on up/down thresholds is widely adopted in commercial IEEE 802.11 devices. Taking the popular ARF algorithm for example, the data rate is increased when ten consecutive transmissions are successful and a date rate downshift is triggered by two consecutive failed transmissions. Although widely deployed, disclosing the implicit objective function that the rate adaptation algorithm is dynamically maximizing, remains as an open problem in the literature. In this paper, we investigate the thresholds-based rate adaptation algorithm via a reverse engineering perspective where the implicit objective function is revealed. We consider this reverse engineering study of the thresholds-based rate adaptation algorithm as an important first step towards a comprehensive understanding on the rate adaptation mechanism designs and the complex interactions among multiple IEEE 802.11 stations.

Wireless ad hoc networks (WANETs) offer communications over a shared wireless channel without any pre-existing infrastructure. Forming peer-to-peer security associations in self-organized WANETs is more challenging than in conventional networks due to the lack of central authorities. In this paper, we propose a generic model to evaluate the relationship of connectivity, memory size, communication overhead and security in fully self-organized WANETs. Based on some reasonable assumptions on node deployment and mobility, we show that when the average number of authenticated neighbors of each node is Theta(1), with respect to the network size n, most of the nodes can be securely connected, forming a connected secure backbone, i.e., the secure network percolates. This connected secure backbone can be utilized to break routing-security dependency loop, and provide enough derived secure links connecting isolated nodes with the secure backbone in a multi-hop fashion, which leads to the secure connectivity of the whole network.

In this paper, we investigate the routing optimization problem in wireless mesh networks. While existing works usually assume static and known traffic demand, we emphasize that the actual traffic is time-varying and difficult to measure. In light of this, we alternatively pursue a stochastic optimiz

This paper addresses the throughput maximization problem in wireless mesh networks. For the case of cooperative access points, we present a negotiation-based throughput maximization algorithm which adjusts the operating channel and power level among access points automatically, from a game-theoretical perspective. We show that this algorithm converges to the optimal channel and power assignment which yields the maximum overall throughput with arbitrarily high probability. Moreover, we analyze the scenario where access points belong to different regulation entities and hence non-cooperative. The long- term behavior and corresponding performance are investigated and the analytical results are verified by simulations.

In this paper, we investigate the channel selection strategy for secondary users in cognitive radio networks. We claim that in order to avoid the costly channel switchings, a secondary user may desire an optimal channel which maximizes the probability of successful transmissions, rather than consistently adapting channels to the random environment. We propose a stochastic channel selection algorithm based on the learning automata techniques. This algorithm adjusts the probability of selecting each available channel and converges to the e-optimal solution asymptotically.

The throughput maximization problem of wireless mesh access networks is addressed. For the case of cooperative access points, we present a negotiation-based throughput maximization algorithm which adjusts the operating frequency and power level among access points autonomously, from a game-theoretical perspective. We show that this algorithm converges to the optimal frequency and power assignment which yields the maximum overall throughput with arbitrarily high probability. Moreover, we analyze the scenario where access points belong to different regulation entities and hence non-cooperative. The long-term behavior and corresponding performance are investigated and the analytical results are verified by simulations.