From Data to Optimization: Data-Free Deep Incremental Hashing With Data Disambiguation and Adaptive Proxies
Authors: 
Qinghang Su, Dayan Wu, Chenming Wu, Bo Li, Weiping WangAuthors Info & Claims
Pages 6576 - 6589
Abstract
Deep incremental hashing methods require a large number of original training samples to preserve old knowledge. However, the old training samples are not always available. This “data-free” setting poses great challenges for learning discriminative codes for new classes (plasticity) and maintaining the code invariance of old ones (stability). On the one hand, the presence of ambiguous data in new-emerging classes, which is highly similar to that in old classes, further aggravates catastrophic forgetting. On the other hand, although well-separated hash codes of new classes can be learned by forcing them towards fixed hash centers, it may significantly change the learned parameters of the old model, leading to severe forgetting on old classes. To alleviate the stability-plasticity dilemma in data-free situations, this paper presents a novel deep incremental hashing method called Data-Free Deep Incremental Hashing (DFIH) from the data to the optimization aspect. We start from the data aspect and propose a data disambiguation module to reveal and discard ambiguous data, especially pixels to alleviate the forgetting issues. Subsequently, we introduce a set of trainable hash proxies during the optimization process. These proxies are optimized adaptively as well as the hash codes, not only guiding the model to learn discriminative hash codes for new classes but also avoiding the dramatic modification of the model’s parameters, thus improving plasticity and maintaining stability. Extensive experiments on six widely-used image retrieval benchmarks and sixteen incremental learning situations show the superiority of DFIH. Ablation analysis further confirms the effectiveness of the components in DFIH. The code of this work is released at <uri>https://github.com/SuQinghang/DFIH</uri>.
References
[1]
Q. Jiang and W. Li, “Asymmetric deep supervised hashing,” in Proc. Conf. Artif. Intell. (AAAI), Apr. 2018, vol. 32, no. 1, pp. 1–8.
[2]
L. Yuanet al., “Central similarity quantization for efficient image and video retrieval,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 3083–3092.
[3]
J. T. Hoe, K. W. Ng, T. Zhang, C. S. Chan, Y.-Z. Song, and T. Xiang, “One loss for all: Deep hashing with a single cosine similarity based learning objective,” in Proc. NIPS, 2021, pp. 24286–24298.
[4]
L. Wang, Y. Pan, C. Liu, H. Lai, J. Yin, and Y. Liu, “Deep hashing with minimal-distance-separated hash centers,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2023, pp. 23455–23464.
[5]
A. Gionis, P. Indyk, and R. Motwani, “Similarity search in high dimensions via hashing,” in Proc. VIDB Conf., San Francisco, CA, USA, Sep. 1999, vol. 99, no. 6, pp. 518–529.
[6]
Y. Weiss, A. Torralba, and R. Fergus, “Spectral hashing,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS), vol. 21, 2008, pp. 1–8.
[7]
Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2916–2929, Dec. 2013.
[8]
D. Wu, Q. Dai, J. Liu, B. Li, and W. Wang, “Deep incremental hashing network for efficient image retrieval,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 9069–9077.
[9]
X. Tian, W. W. Y. Ng, and H. Xu, “Deep incremental hashing for semantic image retrieval with concept drift,” IEEE Trans. Big Data, vol. 9, no. 4, pp. 1102–1115, Jan. 2023.
[10]
Y. Shen, Y. Xiong, W. Xia, and S. Soatto, “Towards backward-compatible representation learning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 6367–6376.
[11]
T. S. T. Wan, J.-C. Chen, T.-Y. Wu, and C.-S. Chen, “Continual learning for visual search with backward consistent feature embedding,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 16681–16690.
[12]
Z. Li and D. Hoiem, “Learning without forgetting,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 12, pp. 2935–2947, Dec. 2017.
[13]
P. Dhar, R. V. Singh, K. Peng, Z. Wu, and R. Chellappa, “Learning without memorizing,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 5138–5146.
[14]
R. French, “Catastrophic forgetting in connectionist networks,” Trends Cognit. Sci., vol. 3, no. 4, pp. 128–135, Apr. 1999.
[15]
M. McCloskey and N. J. Cohen, Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem, vol. 24. Amsterdam, The Netherlands: Elsevier, 1989, pp. 109–165.
[16]
Z. Cao, M. Long, J. Wang, and P. S. Yu, “HashNet: Deep learning to hash by continuation,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 5608–5617.
[17]
B. Liu, Y. Cao, M. Long, J. Wang, and J. Wang, “Deep triplet quantization,” in Proc. 26th ACM Int. Conf. Multimedia, Oct. 2018, pp. 755–763.
[18]
J. Wang, T. Zhang, J. song, N. Sebe, and H. T. Shen, “A survey on learning to hash,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 769–790, Apr. 2018.
[19]
S. Li, X. Li, J. Lu, and J. Zhou, “Structure-adaptive neighborhood preserving hashing for scalable video search,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 4, pp. 2441–2454, Apr. 2022.
[20]
L. Zhu, C. Zheng, W. Guan, J. Li, Y. Yang, and H. T. Shen, “Multi-modal hashing for efficient multimedia retrieval: A survey,” IEEE Trans. Knowl. Data Eng., vol. 36, no. 1, pp. 239–260, Jan. 2024.
[21]
X. Luoet al., “A survey on deep hashing methods,” ACM Trans. Knowl. Discovery Data, vol. 17, no. 1, pp. 1–50, Feb. 2023.
[22]
X. Li, J. Yu, Y. Wang, J.-Y. Chen, P.-X. Chang, and Z. Li, “DAHP: Deep attention-guided hashing with pairwise labels,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 3, pp. 933–946, Mar. 2022. 10.1109/TCSVT.2021.3070129.
[23]
Y. Wang, X. Ou, J. Liang, and Z. Sun, “Deep semantic reconstruction hashing for similarity retrieval,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 1, pp. 387–400, Jan. 2021.
[24]
H. Zhai, S. Lai, H. Jin, X. Qian, and T. Mei, “Deep transfer hashing for image retrieval,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 2, pp. 742–753, Feb. 2021.
[25]
Q. Qin, L. Huang, K. Xie, Z. Wei, C. Wang, and W. Zhang, “Deep adaptive quadruplet hashing with probability sampling for large-scale image retrieval,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 12, pp. 7914–7927, Dec. 2023.
[26]
Y. Shenet al., “Auto-encoding twin-bottleneck hashing,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 2815–2824.
[27]
X. Shen, Y. Tang, Y. Zheng, Y.-H. Yuan, and Q.-S. Sun, “Unsupervised multiview distributed hashing for large-scale retrieval,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 12, pp. 8837–8848, Dec. 2022.
[28]
Q. Qin, L. Huang, Z. Wei, K. Xie, and W. Zhang, “Unsupervised deep multi-similarity hashing with semantic structure for image retrieval,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 7, pp. 2852–2865, Jul. 2021. 10.1109/TCSVT.2020.3032402.
[29]
S. Chenget al., “Uncertainty-aware and multigranularity consistent constrained model for semi-supervised hashing,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 10, pp. 6914–6926, Oct. 2022.
[30]
S. Jinet al., “Ssah: Semi-supervised adversarial deep hashing with self-paced hard sample generation,” in Proc. Conf. Artif. Intell. (AAAI), 2020, vol. 34, no. 7, pp. 11157–11164.
[31]
W. Shi, Y. Gong, B. Chen, and X. Hei, “Transductive semisupervised deep hashing,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 3713–3726, Aug. 2022.
[32]
D. Wu, Q. Dai, B. Li, and W. Wang, “Deep uncoupled discrete hashing via similarity matrix decomposition,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 19, no. 1, pp. 1–22, Jan. 2023.
[33]
H. Lai, Y. Pan, Y. Liu, and S. Yan, “Simultaneous feature learning and hash coding with deep neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 3270–3278.
[34]
D.-W. Zhou, Q.-W. Wang, Z.-H. Qi, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Deep class-incremental learning: A survey,” 2023, arXiv:2302.03648.
[35]
S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “iCaRL: Incremental classifier and representation learning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 2001–2010.
[36]
S. Wang, W. Shi, S. Dong, X. Gao, X. Song, and Y. Gong, “Semantic knowledge guided class-incremental learning,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 10, pp. 5921–5931, Mar. 2023.
[37]
F. Zhu, X.-Y. Zhang, C. Wang, F. Yin, and C.-L. Liu, “Prototype augmentation and self-supervision for incremental learning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2021, pp. 5871–5880.
[38]
W. Shi and M. Ye, “Prototype reminiscence and augmented asymmetric knowledge aggregation for non-exemplar class-incremental learning,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2023, pp. 1772–1781.
[39]
W. Chen, Y. Liu, W. Wang, T. Tuytelaars, E. M. Bakker, and M. S. Lew, “On the exploration of incremental learning for fine-grained image retrieval,” in Proc. Brit. Mach. Vis. Conf. (BMVC), Sep. 2020, pp. 1–12.
[40]
W. W. Y. Ng, X. Tian, Y. Lv, D. S. Yeung, and W. Pedrycz, “Incremental hashing for semantic image retrieval in nonstationary environments,” IEEE Trans. Cybern., vol. 47, no. 11, pp. 3814–3826, Nov. 2017.
[41]
W. W. Y. Ng, X. Tian, W. Pedrycz, X. Wang, and D. S. Yeung, “Incremental hash-bit learning for semantic image retrieval in nonstationary environments,” IEEE Trans. Cybern., vol. 49, no. 11, pp. 3844–3858, Nov. 2019.
[42]
X. Tian, W. W. Y. Ng, H. Wang, and S. Kwong, “Complementary incremental hashing with query-adaptive re-ranking for image retrieval,” IEEE Trans. Multimedia, vol. 23, pp. 1210–1224, 2021.
[43]
R. Elwell and R. Polikar, “Incremental learning of concept drift in nonstationary environments,” IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1517–1531, Oct. 2011.
[44]
S. Zhu, T. Yang, and C. Chen, “Visual explanation for deep metric learning,” IEEE Trans. Image Process., vol. 30, pp. 7593–7607, 2021.
[45]
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 618–626.
[46]
J. Gildenblat and contributors.(2021). Pytorch Library for Cam Methods. [Online]. Available: https://github.com/jacobgil/pytorch-grad-cam
[47]
K. Li, Z. Wu, K. Peng, J. Ernst, and Y. Fu, “Tell me where to look: Guided attention inference network,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Jun. 2018, pp. 9215–9223.
[48]
Y. K. Jang, G. Gu, B. Ko, I. Kang, and N. I. Cho, “Deep hash distillation for image retrieval,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2022, pp. 354–371.
[49]
A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” M.S. thesis, Dept. Comput. Sci., Univ. Toronto, 2009.
[50]
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” in NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[51]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Miami, FL, USA, Jun. 2009, pp. 248–255.
[52]
T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, “NUS-WIDE: A real-world web image database from national university of Singapore,” in Proc. ACM Int. Conf. Image Video Retr., Santorini, Greece, Jul. 2009, pp. 1–9.
[53]
M. J. Huiskes and M. S. Lew, “The MIR Flickr retrieval evaluation,” in Proc. 1st ACM Int. Conf. Multimedia Inf. Retr., Oct. 2008, pp. 39–43.
[54]
Z. Lu, L. Jin, Z. Li, and J. Tang, “Self-paced relational contrastive hashing for large-scale image retrieval,” IEEE Trans. Multimedia, vol. 26, pp. 3392–3404, 2024.
[55]
A. Paszkeet al., “Pytorch: An imperative style, high-performance deep learning library,” in Proc. Adv. Neural Inf. Process. Syst. (NIPS), vol. 32, 2019, pp. 1–12.
[56]
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, May 2017.
[57]
F. Pedregosaet al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, Nov. 2011.
[58]
Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang, “Random erasing data augmentation,” in Proc. AAAI Conf. Artif. Intell., 2020, vol. 34, no. 7, pp. 13001–13008.
Recommendations
Incremental Semi-Supervised classification of data streams via self-representative selection
An illustration of the proposed IS3RS approach.Display Omitted We advance an Incremental Semi-Supervised classification (ISSC) approach via Self-Representative Selection (IS3RS).We develop an incremental self-representative data selection strategy.Most ...
Exemplar-Free Deep Incremental Hashing for Efficient Image Retrieval
Advanced Intelligent Computing Technology and ApplicationsAbstractDeep hashing techniques have been advanced by CNNs’ semantic representations. However, existing incremental hashing methods rely on original data to maintain similarities, which is often inaccessible due to privacy, legal, and transmission ...
POLISH: Adaptive Online Cross-Modal Hashing for Class Incremental Data
WWW '24: Proceedings of the ACM Web Conference 2024In recent years, hashing-based online cross-modal retrieval has garnered growing attention. This trend is motivated by the fact that web data is increasingly delivered in a streaming manner as opposed to batch processing. Simultaneously, the sheer scale ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
1051-8215 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 20 February 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in