We construct a new efficient near duplicate image detection method using a hierarchical hash code learning neural network and load-balanced locality-sensitive hashing (LSH) indexing. We propose a deep constrained siamese hash coding neural network combined with deep feature learning. Our neural network is able to extract effective features for near duplicate image detection. The extracted features are used to construct a LSH-based index. We propose a load-balanced LSH method to produce load-balanced buckets in the hashing process. The load-balanced LSH significantly reduces the query time. Based on the proposed load-balanced LSH, we design an effective and feasible algorithm for near duplicate image detection. Extensive experiments on three benchmark data sets demonstrate the effectiveness of our deep siamese hash encoding network and load-balanced LSH.

Locality-Sensitive Hashing (LSH) is a mainstream method for the Near Duplicate Image Detection (NDID) problem. Previous LSH based methods, however, do not have a principled way to make the indexing structure generate the buckets of similar sizes, which will inevitably degrade the detection effectiveness and efficiency. In this work, we propose a Load-Balanced Locality-Sensitive Hashing (LBLSH) method with a new indexing structure to produce load-balanced buckets for the hashing process. As proved in the paper, the proposed LBLSH can guarantee load-balanced buckets in the hashing process and significantly reduce the query time and the storage space. Based on the proposed LBLSH method, we design an effective and feasible algorithm for the NDID problem. Extensive experiments on two benchmark datasets demonstrate the effectiveness and efficiency of our method.