Abstract
Semantic hashing is an effective technique to empower information retrieval. Currently, considerable efforts have been dedicated to generating high-quality hash codes by modeling document features using generative models and other approaches. However, most of these methods rely solely on a single type of feature, such as TFIDF features, BERT embeddings, etc. As different types of features have distinct but complementary information of documents, e.g. TFIDF mainly contains the keywords information and BERT focuses on the semantics, hash codes generated solely from either may not capture the full essence of the documents. To overcome this challenge, we propose a semantic-alignment-promoting variational auto-encoder to generate hash codes from multiple document features. Specifically, a VAE-based generative model is first developed to model the multiple features. Then, we propose a semantic-alignment-promoting inference network to estimate the parameters of the variational posterior from multiple features. Additionally, the quality of hash codes is further improved by promoting the semantic alignment between the hash codes of connected documents in a constructed connection graph. The results of extensive experiments on three public datasets demonstrate that our proposed model significantly outperforms current state-of-the-art models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14–16, 2014, Conference Track Proceedings (2014)
Chaidaroon, S., Fang, Y.: Variational deep semantic hashing for text documents. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 75–84. SIGIR 2017. Association for Computing Machinery, New York, NY, USA (2017)
Shen, D., et al.: NASH: toward end-to-end neural architecture for generative semantic hashing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2041–2050. Association for Computational Linguistics, Melbourne, Australia, July 2018
Chaidaroon, S., Ebesu, T., Fang, Y.: Deep semantic text hashing with weak supervision. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, pp. 1109–1112. Association for Computing Machinery, New York, NY, USA (2018)
Hansen, C., Hansen, C., Simonsen, J.G., Alstrup, S., Lioma, C.: Unsupervised semantic hashing with pairwise reconstruction. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020, pp. 2009–2012. Association for Computing Machinery, New York, NY, USA (2020)
Stratos, K., Wiseman, S.: Learning discrete structured representations by adversarially maximizing mutual information. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 9144–9154. PMLR, 13–18 July 2020
Ou, Z., Su, Q., Yu, J., Zhao, R., Zheng, Y., Liu, B.: Refining BERT embeddings for document hashing via mutual information maximization. In: Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, Punta Cana, Dominican Republic, November 2021
Qiu, Z., Su, Q., Yu, J., Si, S.: Efficient document retrieval by end-to-end refining and quantizing BERT embedding with contrastive product quantization. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 853–863. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, December 2022
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 1597–1607. PMLR, 13–18 July 2020
Bengio, Y., Léonard, N., Courville, A.C.: Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR abs/1308.3432 (2013)
Dong, W., Su, Q., Shen, D., Chen, C.: Document hashing with mixture-prior generative models. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5226–5235. Association for Computational Linguistics, Hong Kong, China, November 2019
Zheng, L., Su, Q., Shen, D., Chen, C.: Generative semantic hashing enhanced via Boltzmann machines. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 777–788. Association for Computational Linguistics, Online, July 2020
Ye, F., Manotumruksa, J., Yilmaz, E.: Unsupervised few-bits semantic hashing with implicit topics modeling. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 2566–2575. Association for Computational Linguistics, Online, November 2020
Hansen, C., Hansen, C., Simonsen, J.G., Alstrup, S., Lioma, C.: Unsupervised neural generative semantic hashing. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 735–744. SIGIR 2019. Association for Computing Machinery, New York, NY, USA (2019)
Tao, F., et al.: Doc2Cube: allocating documents to text cube without labeled data. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 1260–1265 (2018). https://doi.org/10.1109/ICDM.2018.00169
Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28. Curran Associates, Inc. (2015)
Lehmann, J., et al.: DBpedia - a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web J. 6(2), 167–195 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (No. 62276280, U1811264), Key R &D Program of Guangdong Province (No. 2018B010107005), Natural Science Foundation of Guangdong Province (No. 2021A1515012299).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, J., Su, Q. (2023). Exploiting Multiple Features for Hash Codes Learning with Semantic-Alignment-Promoting Variational Auto-encoder. In: Liu, F., Duan, N., Xu, Q., Hong, Y. (eds) Natural Language Processing and Chinese Computing. NLPCC 2023. Lecture Notes in Computer Science(), vol 14302. Springer, Cham. https://doi.org/10.1007/978-3-031-44693-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-44693-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44692-4
Online ISBN: 978-3-031-44693-1
eBook Packages: Computer ScienceComputer Science (R0)