Deep hashing with multilevel similarity learning for multimedia similarity search

In this work, we propose a novel deep multimodal hashing method, termed as Deep Hashing with Multilevel Similarity Learning (DHMSL), which learns discriminative hash functions with deep neural networks by exploiting multilevel semantic similarity correlations of multimedia data. Firstly, we construct multilevel similarity correlation by jointly exploiting the local structure and semantic label information. Then, the unified binary codes are learned by preserving the multilevel similarity correlations as well as incorporating the bit balance and quantization error properties. Besides that, two deep neural networks are jointly trained to learn two sets of nonlinear hash functions by minimizing the errors of unified binary codes and outputs of the networks. We conduct experiments on two widely-used multimodal datasets, and the proposed DHMSL method can achieve the state-of-the-art performance compared with the baselines for both image-query-text and text-query-image tasks.