In this paper, we present a novel supervised cross-modal hashing framework, namely Scalable disCRete mATrix faCtorization Hashing (SCRATCH). First, it utilizes collective matrix factorization on original features together with label semantic embedding, to learn the latent representations in a shared latent space. Thereafter, it generates binary hash codes based on the latent representations. During optimization, it avoids using a large n × n similarity matrix and generates hash codes discretely. Besides, based on different objective functions, learning strategy, and features, we further present three models in this framework, i.e., SCRATCH-o, SCRATCH-t, and SCRATCH-d. The first one is a one-step method, learning the hash functions and the binary codes in the same optimization problem. The second is a two-step method, which first generates the binary codes and then learns the hash functions based on the learned hash codes. The third one is a deep version of SCRATCH-t, which utilizes deep neural networks as hash functions. The extensive experiments on two widely used benchmark datasets demonstrate that SCRATCH-o and SCRATCH-t outperform some state-of-the-art shallow hashing methods for cross-modal retrieval. The SCRATCH-d also outperforms some state-of-the-art deep hashing models.

The hashing-based approximate nearest neighbors search is able to reduce the storage cost and improve query speed. Therefore, they have attracted much attention in these years. Moreover, some hashing methods have been proposed for cross-modal retrieval tasks. However, there are still some issues that need to be further addressed. For example, some of them only construct a simple similarity matrix when learning hash functions or binary codes, which may lose some useful information. Some of them solve the hard discrete optimization problem by relaxing the binary constraints and quantizing the solution to obtain the final results, which may generate large quantization errors. To address these challenges, we present a new supervised cross-modal hashing method, named supervised robust discrete multimodal hashing (SRDMH). Specifically, it incorporates full label information into the hash functions learning to preserve the similarity in the original space. In addition, instead of relaxing the binary constraints, it is able to learn the binary codes and hash functions simultaneously. Moreover, it adopts a flexible ℓ2,ploss with nonlinear kernel embedding and introduces an intermediate presentation of the binary codes. In light of this, it becomes more robust and easier to solve by an iterative algorithm presented in this paper. To evaluate its performance, we conduct extensive experiments on three benchmark datasets. The results verify that SRDMH outperforms seven state-of-the-art cross-modal hashing methods. In addition, we also extend it to the classification task. Compared with other hashing methods, SRDMH also obtains better results when its binary codes are used for classification.

Recently, deep hashing methods have attracted much attention in multimedia retrieval task. Some of them can even perform cross-modal retrieval. However, almost all existing deep cross-modal hashing methods are pairwise optimizing methods, which means that they become time-consuming if they are extended to large scale datasets. In this paper, we propose a novel tri-stage deep cross-modal hashing method – Dual Deep Neural Networks Cross-Modal Hashing, i.e., DDCMH, which employs two deep networks to generate hash codes for different modalities. Specifically, in Stage 1, it leverages a single-modal hashing method to generate the initial binary codes of textual modality of training samples; in Stage 2, these binary codes are treated as supervised information to train an image network, which maps visual modality to a binary representation; in Stage 3, the visual modality codes are reconstructed according to a reconstruction procedure, and used as supervised information to train a text network, which generates the binary codes for textual modality. By doing this, DDCMH can make full use of inter-modal information to obtain high quality binary codes, and avoid the problem of pairwise optimization by optimizing different modalities independently. The proposed method can be treated as a framework which can extend any single-modal hashing method to perform cross-modal search task. DDCMH is tested on several benchmark datasets. The results demonstrate that it outperforms both deep and shallow state-of-the-art hashing methods.

In recent years, many hashing methods have been proposed for the cross-modal retrieval task. However, there are still some issues that need to be further explored. For example, some of them relax the binary constraints to generate the hash codes, which may generate large quantization error. Although some discrete schemes have been proposed, most of them are time-consuming. In addition, most of the existing supervised hashing methods use an n x n similarity matrix during the optimization, making them unscalable. To address these issues, in this paper, we present a novel supervised cross-modal hashing method---Scalable disCRete mATrix faCtorization Hashing, SCRATCH for short. It leverages the collective matrix factorization on the kernelized features and the semantic embedding with labels to find a latent semantic space to preserve the intra- and inter-modality similarities. In addition, it incorporates the label matrix instead of the similarity matrix into the loss function. Based on the proposed loss function and the iterative optimization algorithm, it can learn the hash functions and binary codes simultaneously. Moreover, the binary codes can be generated discretely, reducing the quantization error generated by the relaxation scheme. Its time complexity is linear to the size of the dataset, making it scalable to large-scale datasets. Extensive experiments on three benchmark datasets, namely, Wiki, MIRFlickr-25K, and NUS-WIDE, have verified that our proposed SCRATCH model outperforms several state-of-the-art unsupervised and supervised hashing methods for cross-modal retrieval.

Recently, hashing based approximate nearest neighbor search has attracted much attention in large scale data search task. Moreover, some cross-modal hashing methods have also been proposed to perform efficient search of different modalities. However, there are still some problems to be further considered. For example, some of them cannot make use of label information, which contains helpful information to generate hash codes; some of them firstly relax binary constraints during optimization, then threshold continuous outputs to binary, which could generate large quantization error. To consider these problems, in this paper, we propose a supervised cross-modal hashing without relaxation (SCMH-WR). It can not only make use of label information, but also generate the final binary codes directly, i.e., without relaxing binary constraints. Specifically, it maps different modalities into a common low-dimension subspace with preserving the similarity of labels; at the same time, it learns a rotation matrix to minimize the quantization error and gets the final binary codes. In addition, an iterative algorithm is proposed to tackle the optimization problem. SCMH-WR is tested on three benchmark data sets. Experimental results demonstrate that SCMH-WR outperforms state-of-the-art hashing methods for cross-modal search task.

Recently, some cross-modal hashing methods have been devised for cross-modal search task. Essentially, given a similarity matrix, most of these methods tackle a discrete optimization problem by separating it into two stages, i.e., first relaxing the binary constraints and finding a solution of the relaxed optimization problem, then quantizing the solution to obtain the binary codes. This scheme will generate large quantization error. Some discrete optimization methods have been proposed to tackle this; however, the generation of the binary codes is independent of the features in the original space, which makes it not robust to noise. To consider these problems, in this paper, we propose a novel supervised cross-modal hashing method---Semi-Relaxation Supervised Hashing (SRSH). It can learn the hash functions and the binary codes simultaneously. At the same time, to tackle the optimization problem, it relaxes a part of binary constraints, instead of all of them, by introducing an intermediate representation variable. By doing this, the quantization error can be reduced and the optimization problem can also be easily solved by an iterative algorithm proposed in this paper. Extensive experimental results on three benchmark datasets demonstrate that SRSH can obtain competitive results and outperform state-of-the-art unsupervised and supervised cross-modal hashing methods.