Because of fast retrieval speed and low storage cost, deep hash learning has become a current research spot in multimedia retrieval, which arouses more and more interest and attention. Most of the available deep hashing methods usually employ the mini-batch strategy to construct training batches, and only one mini-batch of samples is valuable at each iteration during model optimization, which fails to explore the original neighbor structure well, leading to sub-optimal embeddings, especially for relatively large datasets. By contrast, the superior performance is achieved by optimizing the SoftMax loss for certain hashing learning, meanwhile, prior research has suggested the normalized SoftMax loss is essentially identical to one smoothed triplet-wise function, in which each class is assigned to one single semantic center. Nevertheless, under real-world scenarios, one class could correspond to multiple local clusters rather than just a single one. To this end, by extending SoftMax loss with multiple semantic centers, a novel deep hashing framework, called Deep Neighbor Discriminant Binary Embedding (NDBE) framework, is presented to generate discriminative hash codes with the original neighbor structure preservation. Specifically, by expanding the size of the last fully connected layer to increase multiple centers for each class, a novel SoftTriple loss is proposed to capture the latent semantic distribution of the original samples and reduce the intra-class variance, which is optimized without constraints sampling. To learn the different numbers of each class center, the class-center aggregation strategy is developed to obtain the compact set of centers while maintaining semantic similarity. Extensive experiments on three public multi-label datasets show that our proposed NDBE framework achieves superior visual similarity search performance over several state-of-the-art approaches. The source code for the implementation of our proposed NDBE framework is available athttps://github.com/QinLab-WFU/NDBE.