Popular big data frameworks commonly run atop Java Virtual Machine (JVM) and rely on garbage collection (GC) mechanism to automatically allocate/reclaim in-memory objects. Existing garbage collectors are designed based on the hypothesis that most objects are short lived. However, big data frameworks usually generate many long-lived data objects, which can cause heavy GC overhead. Recent approaches have reduced GC overhead in big data frameworks but still suffer from heavy human efforts, additional runtime overhead, or suboptimal GC efficiency. This article describes the design of BridgeGC , a big-data-friendly garbage collector that significantly reduces GC overhead introduced by long-lived data objects. BridgeGC follows a cross-level co-design. At the big data framework level, BridgeGC provides two annotations for framework developers to denote the creation and release of data objects. Based on the annotations, BridgeGC tracks the lifecycles of annotated data objects and optimizes their allocation/reclamation at the GC level. At the GC level, we design a label-based allocator that stores data objects separately from other objects and balances their memory usage in the same JVM, leading to fewer GC cycles. We further design an efficient collector to eliminate unnecessary marking and copying of data objects during GC cycles, lowering the GC time. We have integrated BridgeGC into OpenJDK ZGC. The extensive evaluation, using two popular big data frameworks (Flink and Spark) and a key–value database (Cassandra), shows that BridgeGC achieves 31–82% GC time reduction compared to the baseline ZGC. BridgeGC also outperforms other traditional and academic garbage collectors in end-to-end performance.

Graph algorithms are widely applied in social networks, computational biology, Internet security and a broad range of complexity science. Although there are many state-of-the-art graph frameworks, few frameworks support parallel graph mining in joint cloud computing environment. In this paper, we propose GraphLib, a parallel graph mining library, based on a BSP (Bulk Synchronous Parallel) service over joint cloud computing which was proposed in our prior work. We first summarize the features of commonly-used graph mining algorithms, and present our approaches for parallelizing typical graph mining algorithms. GraphLib includes 17 parallel graph mining algorithms that can be used in 3 scenarios. We evaluate the performance of 4 typical parallel graph algorithms in GraphLib on three real-world datasets. Our parallelized algorithms can achieve sub-linear scalability.

This paper deals with the geometric multi-model fitting from noisy, unstructured point set data (e.g., laser scanned point clouds). We formulate multi-model fitting problem as a sequential decision making process. We then use a deep reinforcement learning algorithm to learn the optimal decisions towards the best fitting result. In this paper, we have compared our method against the state-of-the-art on simulated data. The results demonstrated that our approach significantly reduced the number of fitting iterations.

This paper presents a novel method for reconstruction of three-dimensional (3D) zebra crossings from mobile laser scanning (MLS) point clouds. Firstly, we extract the zebra crossings from the 3D point cloud data in data preprocessing. Secondly, the fitting model is generated by seven parameters to determine one plane commonly and then calculating similarity for fitting the zebra crossings point clouds. Finally, the cuckoo search algorithm is used to adjust the parameters and optimize the results to obtain the optimal model and the geometric information of the zebra crossings can be acquired simultaneously. The proposed algorithm is tested on a set of point-clouds acquired by a RIEGL VMX-450 LiDAR system. The experimental results show the feasibility and stability of our method and the zebra crossings of urban road can be automatically and effectively reconstructed.