In recent years, cross-modal hashing (CMH) has attracted considerable attention due to its ability to learn across different modalities and its high efficiency for similarity retrieval applications. This procedure is computationally inexpensive when dealing with large-scale multi-modalities datasets. However, they do not form the ideal representative model to fully exploit multi-modal data’s underlying properties despite their successful performance. We identify that: (i) most CMH models in their current forms transform the real data points into discrete compact binary codes, which can limit their ability to prevent the loss of important information and thereby produce suboptimal results. (ii) the discrete-binary constraint model is hard to implement, and relaxing the binary constraints is a common property in most existing methods, which often leads to significant quantization errors. (iii) handling the CMH in a symmetry domain leads to a complex and inefficient optimization problem. This paper addresses the above challenges and proposes a novel Asymmetric Correlation Quantization Hashing (ACQH) method. ACQH learns a projection matrix for each heterogeneous modality to map the data point into a low-dimensional semantic space and constructs a compositional quantization to generate hash codes, using the pairwise semantic similarity preservation and the pointwise label regression. As a specific instantiation of our model, we use discrete iterative optimization to obtain the unified hash codes across different modalities. Extensive experiments show that ACQH outperforms state-of-the-art methods on several diverse datasets.