Distance-Based Class Activation Map for Metric Learning
Pages 336 - 347
Abstract
The interpretability of deep neural networks can serve as reliable guidance for algorithm improvement. By visualizing class-relevant features in the form of heatmap, the Class Activation Map (CAM) and derivative versions have been widely exploited to study the interpretability of softmax-based neural networks. However, CAM cannot be adopted directly for metric learning, because there is no fully-connected layer in metric-learning-based methods. To solve this problem, we propose a Distance-based Class Activation Map (Dist-CAM) in this paper, which can be applied to metric learning directly. Comprehensive experiments are conducted with several convolutional neural networks trained on the ILSVRC 2012 and the result shows that Dist-CAM can achieve better performance than the original CAM in weakly-supervised localization tasks, which means the heatmap generated by Dist-CAM can effectively visualize class-relevant features. Finally, the applications of Dist-CAM on specific tasks, i.e., few-shot learning, image retrieval and re-identification, based on metric learning are presented.
References
[1]
Cakir, F., He, K., Xia, X., Kulis, B., Sclaroff, S.: Deep metric learning to rank. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1861–1870 (2019)
[2]
Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839–847 (2018)
[3]
Chu, W., Wang, Y.F.: Learning semantics-guided visual attention for few-shot image classification. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 2979–2983 (2018).
[4]
Ge, W., Huang, W., Dong, D., Scott, M.R.: Deep metric learning with hierarchical triplet loss. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 272–288 (2018)
[5]
Hao, Y., Wang, N., Li, J., Gao, X.: Hsme: Hypersphere manifold embedding for visible thermal person re-identification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 8385–8392 (2019)
[6]
Li, X., et al.: Learning to self-train for semi-supervised few-shot classification. In: 33rd Conference on Neural Information Processing Systems. vol. 32, pp. 10276–10286 (2019)
[7]
Liu, J., Song, L., Qin, Y.: Prototype rectification for few-shot learning. In: ECCV, vol. 1. pp. 741–756 (2019)
[8]
Liu, L., Zhou, T., Long, G., Jiang, J., Yao, L., Zhang, C.: Prototype propagation networks (PPN) for weakly-supervised few-shot learning on category graph. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 3015–3022 (2019)
[9]
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6738–6746 (2017)
[10]
Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W.: Bag of tricks and a strong baseline for deep person re-identification. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 0–0 (2019)
[11]
Rusu, A.A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., Hadsell, R.: Meta-learning with latent embedding optimization. In: International Conference on Learning Representations (2018)
[12]
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, and Batra D Grad-CAM: visual explanations from deep networks via gradient-based localization Int. J. Comput. Vis. 2020 128 2 336-359
[13]
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations 2015 (ICLR 2015) (2015)
[14]
Snell J, Swersky K, and Zemel RS Prototypical networks for few-shot learning Adv. Neural Inf. Process. Syst. 2017 30 4077-4087
[15]
Song, H.O., Jegelka, S., Rathod, V., Murphy, K.: Deep metric learning via facility location. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2206–2214 (2017)
[16]
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Tech. Rep. CNS-TR-2011-001, California Institute of Technology (2011)
[17]
Wang, H., Zhu, X., Xiang, T., Gong, S.: Towards unsupervised open-set person re-identification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 769–773 (2016).
[18]
Wang, J., Zhou, F., Wen, S., Liu, X., Lin, Y.: Deep metric learning with angular loss. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2612–2620 (2017)
[19]
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: IEEE International Conference on Computer Vision (2015)
[20]
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society (2016)
[21]
Zhou, Y., Zhu, Y., Ye, Q., Qiu, Q., Jiao, J.: Weakly supervised instance segmentation using class peak response. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3791–3800 (2018)
Index Terms
- Distance-Based Class Activation Map for Metric Learning
Index terms have been assigned to the content through auto-classification.
Recommendations
Video-based kinship verification using distance metric learning
We investigate the problem of kinship verification from facial videos.We present a new video face dataset for the video-based kinship verification study.We develop a benchmark to evaluate state-of-the-art metric learning methods in video-based kinship ...
Multiple metric learning via local metric fusion
AbstractAdaptive distance metric learning based on the characteristics of data can significantly improve the learner’s performance. Due to the limitations of single metric learning for heterogeneous data, multiple local metric learning has ...
Transfer metric learning by learning task relationships
KDD '10: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data miningDistance metric learning plays a very crucial role in many data mining algorithms because the performance of an algorithm relies heavily on choosing a good metric. However, the labeled data available in many applications is scarce and hence the metrics ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Oct 2021
593 pages
ISBN:978-3-030-88012-5
DOI:10.1007/978-3-030-88013-2
© Springer Nature Switzerland AG 2021.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 October 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025