In this work, we propose B 3 -tree , a hybrid index for persistent memory that leverages the byte-addressability of the in-memory index and the page locality of B-trees. As in the byte-addressable in-memory index, B 3 -tree is updated by 8-byte store instructions. Also, as in disk-based index, B 3 -tree is failure-atomic since it makes every 8-byte store instruction transform a consistent index into another consistent index without the help of expensive logging. Since expensive logging becomes unnecessary, the number of cacheline flush instructions required for B 3 -tree is significantly reduced. Our performance study shows that B 3 -tree outperforms other state-of-the-art persistent indexes in terms of insert and delete performance. While B 3 -tree shows slightly worse performance for point query performance, the range query performance of B 3 -tree is 2x faster than FAST and FAIR B-tree because the leaf page size of B 3 -tree can be set to 8x larger than that of FAST and FAIR B-tree without degrading insertion performance. We also show that read transactions can access B 3 -tree without acquiring a shared lock because B 3 -tree remains always consistent while a sequence of 8-byte write operations are making changes to it. As a result, B 3 -tree provides high concurrency level comparable to FAST and FAIR B-tree.

We present EclipseMR, a novel MapReduce framework prototype that efficiently utilizes a large distributed memory in cluster environments. EclipseMR consists of double-layered consistent hash rings - a decentralized DHT-based file system and an in-memory key-value store that employs consistent hashing. The in-memory key-value store in EclipseMR is designed not only to cache local data but also remote data as well so that globally popular data can be distributed across cluster servers and found by consistent hashing. In order to leverage large distributed memories and increase the cache hit ratio, we propose a locality-aware fair (LAF) job scheduler that works as the load balancer for the distributed in-memory caches. Based on hash keys, the LAF job scheduler predicts which servers have reusable data, and assigns tasks to the servers so that they can be reused. The LAF job scheduler makes its best efforts to strike a balance between data locality and load balance, which often conflict with each other. We evaluate EclipseMR by quantifying the performance effect of each component using several representative MapReduce applications and show EclipseMR is faster than Hadoop and Spark by a large margin for various applications.