Low-latency data service is an increasingly critical challenge for data center applications. In modern distributed storage systems, proper data placement helps reduce the data movement delay, which can contribute to the service latency reduction tremendously. Existing data placement solutions have often assumed the prior distribution of data requests or discovered it via trace analysis. However, data placement is a difficult online decision-making problem faced with dynamic network conditions and time-varying user request patterns. The conventional static model-based solutions are less effective to handle the dynamic system. With an overall consideration of data movement and analytical latency, we develop a reinforcement learning-based framework DataBot+, automatically learning the optimal placement policies. DataBot+ adopts neural networks, trained with a variant of$Q$-learning, whose input is the real-time data flow measurements and whose output is a value function estimating the near-future latency. For instantaneous decision making, DataBot+ is decoupled into two asynchronous production and training components, ensuring that the training delay will not introduce extra overheads to handle the data flows. Evaluation results driven by real-world traces demonstrate the effectiveness of our design.

Infrastructure-as-a-Service clouds in geo-distributed data centers offer various pricing options, including on-demand and reserved instances, which provide an elastic and cost-effective infrastructure to support High Performance Computing (HPC) applications. In this paper, we propose an instance reservation based cloud service framework, modeling the cost-minimizing reservation decision issue as an NP-hard integer programming problem for distributed data centers. To ease its computation complexity, two algorithms are proposed to minimize the HPC service cost with the worst-case performance guarantees: an offline heuristic-greedy algorithm, and a rolling-horizon based online algorithm when only short-term demand prediction is available. Facing fluctuating demands, instance reservation in a single data center may incur the highly underutilized capacity. To address this issue for further cost reduction, we extend the scheme with a novel cloud broker federation based resource sharing mechanism, reallocating already reserved but unused instances to computation-intensive and short-lived tasks for continuous execution without interruption. Extensive evaluations driven by large-scale trace-based datasets demonstrate that the proposed mechanism can effectively handle large volumes of service requests, saving considerable service costs with higher reservation resource utilization.

Data-intensive applications need to address the problem of properly placing the set of data items in geo-distributed storage nodes. Traditional techniques use the hashing method to achieve the load balance among nodes such as those used in Hadoop and Cassandra, but are not efficient for the requests reading multiple data items in one transaction, especially when the source locations of requests are also distributed. Some recent papers proposed the managed data placement schemes for online social networks, but have a limited scope of applications due to their focuses. We propose a general hypergraph-based data placement framework, which considers both the performance metrics related to the co-location of associated data and those related to the exact location of fulfilling each requested data item. In the framework, we present the methods to convert the optimization objectives into hypergraph models and employ a hypergraph partitioning to efficiently partition the set of data items and place them in distributed nodes. Further, we extend the scheme into replica placement where we need to find multiple locations to place the replicas of the same data item. Through extensive experiments based on trace-based datasets, we evaluate the performance of the proposed framework and demonstrate its effectiveness.

Low-latency data access is an important challenge for data center networks. Proper placement of the data items can reduce the data travel time in the distributed storage systems, which contributes significantly to the latency reduction. Most existing data placement approaches have often assumed the prior distribution of data requests or discovered so through trace analysis. However, the traditional static model-based solutions are less effective to handle the system uncertainties in a dynamic environment. We present DataBot, a reinforcement learning-based adaptive framework, to learn the optimal data placement policies faced with the dynamic network conditions and time-varying request patterns. DataBot utilizes a neural network, trained with a variant of Q-learning, whose input is the realtime data flow measurements and whose output is a value function estimating the near-future latency. For rapid decision making, DataBot is divided into two decoupled production and training components, ensuring that the convergence time of the training will not introduce more overheads to serve the read/write requests. Evaluation results demonstrate that the average write and read latency of the whole system can be lowered by about 35% and 40%, respectively.

The distributed cache system contains a group of servers caching different contents based on consistent hashing. The dynamic provisioning of servers helps to improve the system efficiency, which leads to a reduction of energy cost. We first measure the cache hit rate, request batching effect and cache warm-up time of the system through experiments, considering that they can affect the system performance and efficiency. Then we formulate a stochastic network optimization problem, which aims at achieving objectives on the queue stability, energy cost and cache hit rate simultaneously, through the dynamic control of server activeness and request dispatching. The problem is transformed into a minimization problem in each time slot, which is further addressed through the proposed efficient online algorithm based on dynamic programming. Moreover, we improve the scheme with several practical considerations in the scheme implementation. Finally, the proposed algorithm and the improvements are evaluated through extensive experiments.

Delay-Tolerant Networks (DTNs) are wireless mobile networks, where the nodes are sparse and end-to-end connectivity is rare. Since DTN nodes are mostly energy-limited devices, there is an immediate need to have energy-efficient routing protocols, allowing the network to perform better and function longer. Besides, in the real world, people carrying the nodes form a lot of communities because of similar interests, and they behave with social selfishness. How to improve the energy efficiency in multi-community scenarios has been an important problem. In this paper, we analytically model the performance of epidemic routing protocols in multi-community scenarios with social selfishness considerations using the Ordinary Differential Equations (ODEs). Further, an energy-efficient copy-limit-optimized algorithm based on the Box's complex method for epidemic routing is proposed, which is designed to determine the optimal copy limit in multiple communities, and can improve the energy efficiency effectively. At last, both the numerical and simulation results show that the routing protocol with the proposed algorithm can reduce the energy consumption effectively, and the impact of social selfishness is also analyzed.

With the increasing demand of big data applications, a variety of problems on how to operate the supporting infrastructures more intelligently and efficiently have attracted much attention in the literature. To optimize the data placement among distributed network locations is one of the fundamental problems, which aims at facilitating the data storage and access. However, traditional schemes meet challenges on the running time and the overhead introduced due to the increasing scale of datasets. Therefore, we propose a novel data placement scheme based on sketches to overcome these challenges. We first justify the effectiveness of applying the hypergraph sparsification on the data placement problem, and then present the method of constructing sparsifiers through the sketches of request traffic. Besides, the scheme features on the support of aggregating distributed sketches to make the decision and capturing the pattern of recent traffic through sliding windows. Finally, we obtain numerical results through simulations which confirm that the proposed scheme can place data effectively while reducing the introduced overhead in terms of algorithm running time, space and network traffic.

Data-intensive applications need to address the problem of how to properly place the set of data items to distributed storage nodes. Traditional techniques use the hashing method to achieve the load balance among nodes such as those in Hadoop and Cassandra, but they do not work efficiently for the requests reading multiple data items in one transaction, especially when the source locations of requests are also distributed. Recent works proposed the managed data placement schemes for online social networks, but have a limited scope of applications due to their focuses. We propose an associated data placement (ADP) scheme, which improves the co-location of associated data and the localized data serving while ensuring the balance between nodes. In ADP, we employ the hypergraph partitioning technique to efficiently partition the set of data items and place them to the distributed nodes, and we also take replicas and incremental adjustment into considerations. Through extensive experiments with both synthesized and trace-based datasets, we evaluate the performance of ADP and demonstrate its effectiveness.

Information-Centric Networking (ICN) has a great potential to better support the content distribution over the future Internet. Meanwhile, users with mobile devices will also access ICN, introducing a new challenge to ICN providers to handle such mobile content requests. In this paper, a mobility and popularity-based caching strategy is proposed to increase the cache hit rate through WiFi while people are moving from one WiFi hotspot to another. It thus lowers the network traffic from the local ICN to the outside Internet. Specifically, a precise derivation on the content request rate is provided based on the content popularity and user mobility. A novel caching strategy is further developed based on this derivation for both the mobility-oblivious and mobility-aware scenarios. We evaluate different caching strategies with a trace-driven simulation and our new approach can achieve a 33.84% reduction on the network traffic when compared with the caching strategies only considering the content popularity.

Geocast, delivering messages to a specific location, has become an important issue with the accelerated development of the location-based services in mobile networks. Geocast in the automotive domain is of particular interest, enabling many promising applications, such as geographic advertising, location-based traffic alerts, etc. Different from the conventional geocast algorithms focusing on the distance-based approaches, in this paper, we propose a mobility-aware geocast algorithm (GeoMob) for urban VANETs from the Delay-Tolerant Network (DTN) perspective to better deal with the high mobility and transient connectivity issues. Different levels and aspects of vehicle mobility information are employed, making GeoMob very simple, scalable and communication and compunction-effective. Practical issues are well considered by introducing real-world trace analysis, trace-driven simulation and efficient buffer management. Extensive performance comparisons with other protocols have been conducted to show the advantages of GeoMob.

The dynamic provisioning of distributed cache services helps to improve the system efficiency. We model the system as groups of servers caching different and none-overlapping key segments of content objects, and investigate the benefit of cache hit and request batching. A stochastic network optimization problem is formulated, which aims at achieving the system stability, low energy cost and certain cache hit rate simultaneously through the dynamic control of server activeness and request dispatching. The problem is transformed into a minimization problem at each time slot and the online algorithm to solve it is proposed. Also we show that dynamic programming helps to lower the computational complexity. Finally, the proposed algorithm is evaluated through extensive simulations.

The user-aware content retrieval services are always data-intensive and require much resource to satisfy the user demand, which incurs the high cost of implementation. Considering that they could largely exploit the pay-as-you-go paradigm and the almost unlimited resource pool of the cloud, we investigate the design issues of deploying the services to the cloud in a cost-effective way. We formulate the resource allocation and request handling problem which aims at lowering the deployment cost and guaranteeing the service quality simultaneously. Due to the hardness of obtaining an optimal solution, we design two approximate algorithms with different points of emphasis and analyze their approximation ratios as well. In addition, we discuss the implementation issues in applying the proposed algorithms to the practical systems. Finally, the algorithms are evaluated and validated through both trace-based and synthesized simulations where they show a large improvement in terms of the total system cost.