Distributed key-value stores provide the Multiget API, where many key-value operations are batched together, to meet the parallel requirement of applications. Correspondingly, reducing the latency of Multigets is crucial for the responsiveness of the distributed key-value stores. The latency of a Multiget depends on both which replica server its key-value operations are scheduled to, i.e., the replica selection for each key-value operation, and when these key-value operations are served, i.e., the scheduling of the service sequence at different replica servers. Existing solutions solely focus on either one of them and accordingly lead to suboptimal latency for Multigets. To address these issues, this paper proposes an Adaptive Multiget Scheduling (AMS) algorithm in this paper, and specifically, our AMS re-architectures the framework to remove the conflict between replica selection and service sequence scheduling in the existing solution Rein. Based on the new framework, a sophisticated replica selection method is designed. Furthermore, AMS guides both replica selection and service sequence scheduling by the piggybacked information of replica servers, being adaptive to the heterogeneous time-varying server performance. Consequently, AMS can respectively reduce the median,$95^{th}$, and$99^{th}$percentile latencies of Multigets by a factor of 4, 3.1, and 1.86 compared to the default FIFO algorithm and significantly outperforms Rein.

Nowadays, the distributed key-value stores have become the basic building block for large-scale cloud applications. In large-scale distributed key-value stores, many key-value access operations, which will be processed in parallel on different servers, are usually generated for a single end-user request. Accordingly, the completion time of an end-user request is determined by the last completed key-value access operation. Scheduling the order of serving key-value access operations can effectively reduce the completion times of end requests, thereby improving the user experience. However, existing scheduling algorithms hardly achieve consistent low latency due to the following challenges: the large overhead of cooperating clients and servers, the time-varying load and performance of servers, the traffic distribution can be either heavy-tailed or light-tailed and both the mean and the tail completion time are expected to be low. In this paper, we formalize the problem of scheduling key-value access operations and show it is NP-hard. Furthermore, we heuristically design the distributed adaptive scheduler (DAS), which distributively combines the largest remaining processing time last and the shortest remaining process time first algorithms. Theoretical analysis shows that DAS is adaptive to the time-varying traffic and server performance and can achieve consistent low mean and tail latency regardless of traffic distributions. Extensive simulations show that DAS reduces the mean request completion time by$17 \! \sim \! 50\%$with heavy-tailed traffic and$2 \! \sim 26 \! \%$with light-tailed traffic, while keeping the smallest tail completion time, compared to the default first come first served algorithm. Moreover, DAS outperforms the existing Rein-SBF algorithm under various scenarios.

In distributed key-value stores, multiple replica servers are always available for each key-value access operation when the eventual consistency model is employed. Accordingly, the completion times of the key-value access operations generated by an end-user request at different servers may be of great difference, especially when the replica servers are heterogeneous and have time-varying performance. Accordingly, the replica selection algorithm is crucial to cut the response time of end-user requests. The main challenge of making replica selection for each light-weighted key-value access operation is to timely know the status of replica serves. Recently, the adaptive replica selection algorithm C3 suggests guiding the replica selection with the piggybacked information of replica server in the returned “value”. Although C3 has good performance, the poor timeliness of feedback information makes a large performance gap between C3 and the ideal replica selection algorithm. To narrow this gap, the Accelerated Information Dissemination (AID) mechanism is proposed in this paper. Specifically, AID removes the bottleneck of information dissemination at the “slow” servers by letting both “client” and “replica server” store the records about the status of replica servers and both “key” and “value” piggyback multiple records. AID is implemented in Cassandra and evaluated by experiments and large scale simulations. The results show AID can significantly improve the timeliness of feedback information, especially when the number of nodes is large. Accordingly, AID helps C3 to greatly reduce the latency.

Nowadays, the distributed key-value stores have become the basic building block for large scale cloud applications. In large-scale distributed key-value stores, many key-value access operations, which will be processed in parallel on different servers, are usually generated for the data required by a single end-user request. Hence, the completion time of the end request is determined by the last completed key-value access operation. Accordingly, scheduling the order of key-value access operations of different end requests can effectively reduce their completion time, improving the user experience. However, existing algorithms are either hard to employ in distributed key-value stores due to the relatively large cooperation overhead for centralized information or unable to adapt to the time-varying load and server performance under different traffic patterns. In this paper, we first formalize the scheduling problem for small mean request completion time. As a step further, because of the NP-hardness of this problem, we heuristically design the distributed adaptive scheduler (DAS) for distributed key-value stores. DAS reduces the average request completion time by a distributed combination of the largest remaining processing time last and shortest remaining process time first algorithms. Moreover, DAS is adaptive to the time-varying server load and performance. Extensive simulations show that DAS reduces the mean request completion time by more than 15 ~ 50% compared to the default first come first served algorithm and outperforms the existing Rein-SBF algorithm under various scenarios.

In distributed key-value stores, multiple replica servers are always available for each key-value access operation when the eventually consistency model is employed. Accordingly, the replica selection algorithm is crucial to the tail latency of the key-value access operations generated by end-user requests, especially under the environment of heterogeneous replica servers. The main challenge of making replica selection decision is to know the status of replica serves with acceptable overhead for each lightweight key-value access operation. Recently, aware of the time-varying performance and load of replica servers, the adaptive replica selection algorithm C3 suggests piggybacking the information of replica server via the returned value” to guide the replica selection. Although the good performance of C3 has been verified by experiments and simulations, the poor timeliness of feedback information makes a large performance gap between C3 and the ideal replica selection algorithm. To narrow this gap, we propose the information dissemination mechanism for C3, which lets both “client” and “server” store the records about the status of replica servers, and both key” and value” piggyback multiple records. In this way, the information dissemination bottleneck at “slow” server would be removed with the help of multiple clients and servers. As confirmed by simulation results, the information dissemination mechanism can significantly improve the timeliness of feedback information with acceptable overhead and accordingly helps C3 to greatly reduce the tail latency (by about 35% under the default scenario).