Abstract
Understanding the behavior of deep networks is crucial to increase our confidence in their results. Despite an extensive body of work for explaining their predictions, researchers have faced reliability issues, which can be attributed to insufficient formalism. In our research, we adopt novel probabilistic and spectral perspectives to formally analyze explanation methods. Our study reveals a pervasive spectral bias stemming from the use of gradient, and sheds light on some common design choices that have been discovered experimentally, in particular, the use of squared gradient and input perturbation. We further characterize how the choice of perturbation hyperparameters in explanation methods, such as SmoothGrad, can lead to inconsistent explanations and introduce two remedies based on our proposed formalism: (i) a mechanism to determine a standard perturbation scale, and (ii) an aggregation method which we call SpectralLens. Finally, we substantiate our theoretical results through quantitative evaluations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Also note that as we are using real valued functions, other components i.e. real odd and imaginary even, are zero.
- 2.
Rect function is defined as \(\operatorname {Rect}(\tilde{\boldsymbol{x}})=\frac{1}{2}\operatorname {sign}(\frac{1}{2}-|\tilde{\boldsymbol{x}}|)+\frac{1}{2}\).
References
Food-101 – Mining Discriminative Components with Random Forests. https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity Checks for Saliency Maps (2020). https://doi.org/10.48550/arXiv.1810.03292, [cs, stat]
Agarwal, S., Jabbari, S., Agarwal, C., Upadhyay, S., Wu, Z.S., Lakkaraju, H.: Towards the Unification and Robustness of Perturbation and Gradient Based Explanations (2021). https://doi.org/10.48550/arXiv.2102.10618, [cs]
Alvarez-Melis, D., Jaakkola, T.S.: On the Robustness of Interpretability Methods (2018). https://doi.org/10.48550/arXiv.1806.08049, [cs, stat]
Ancona, M., Ceolini, E., Öztireli, C., Gross, M.: Towards better understanding of gradient-based attribution methods for Deep Neural Networks (2018). https://doi.org/10.48550/arXiv.1711.06104, [cs, stat]
Arrieta, A.B., et al.: Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019). https://doi.org/10.48550/arXiv.1910.10045, [cs]
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), e0130140 (2015). https://doi.org/10.1371/journal.pone.0130140. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130140
Balduzzi, D., et al.: The Shattered Gradients Problem: if resnets are the answer, then what is the question? (2018). https://doi.org/10.48550/arXiv.1702.08591, [cs, stat]
Bansal, N., Agarwal, C., Nguyen, A.: SAM: The Sensitivity of Attribution Methods to Hyperparameters (2020). https://doi.org/10.48550/arXiv.2003.08754, [cs]
Basri, R., Galun, M., Geifman, A., Jacobs, D., Kasten, Y., Kritchman, S.: Frequency Bias in Neural Networks for Input of Non-Uniform Density (2020). https://api.semanticscholar.org/CorpusID:212644664
Benbarka, N., Hofer, T., Ul-Moqeet Riaz, H., Zell, A.: Seeing implicit neural representations as Fourier series. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2283–2292 (2022). https://doi.org/10.1109/WACV51458.2022.00234. https://ieeexplore.ieee.org/document/9706795/
Blackman, R.B., Tukey, J.W.: The measurement of power spectra from the point of view of communications engineering – Part I. Bell Syst. Tech. J. 37(1), 185–282 (1958). https://doi.org/10.1002/j.1538-7305.1958.tb03874.x. http://ieeexplore.ieee.org/document/6768513/
Brughmans, D., Melis, L., Martens, D.: Disagreement amongst counterfactual explanations: how transparency can be deceptive (2023). https://doi.org/10.48550/arXiv.2304.12667, [cs]
Bykov, K., Hedström, A., Nakajima, S., Höhne, M.M.C.: NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights (2022). https://doi.org/10.48550/arXiv.2106.10185, [cs]
Cao, Y., Fang, Z., Wu, Y., Zhou, D.X., Gu, Q.: Towards understanding the spectral bias of deep learning. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, Canada, pp. 2205–2211. International Joint Conferences on Artificial Intelligence Organization (2021). https://doi.org/10.24963/ijcai.2021/304. https://www.ijcai.org/proceedings/2021/304
Chen, H., Janizek, J.D., Lundberg, S., Lee, S.I.: True to the Model or True to the Data? (2020). https://doi.org/10.48550/arXiv.2006.16234, [cs, stat]
Chen, J., Song, L., Wainwright, M.J., Jordan, M.I.: Learning to Explain: An Information-Theoretic Perspective on Model Interpretation (2018). https://doi.org/10.48550/arXiv.1802.07814, [cs, stat]
Cohen, L.: The generalization of the Wiener-Khinchin theorem. In: Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 1998 (Cat. No. 98CH36181), vol. 3, pp. 1577–1580. IEEE (1998). https://ieeexplore.ieee.org/abstract/document/681753/
Covert, I., Lundberg, S., Lee, S.I.: Explaining by Removing: A Unified Framework for Model Explanation (2022). https://doi.org/10.48550/arXiv.2011.14878, [cs, stat]
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848. https://ieeexplore.ieee.org/document/5206848. iSSN 1063-6919
Dosovitskiy, A., et al.: An Image is Worth \(16 \times 16\) Words: Transformers for Image Recognition at Scale (2021). https://doi.org/10.48550/arXiv.2010.11929, [cs]
Fong, R., Patrick, M., Vedaldi, A.: Understanding Deep Networks via Extremal Perturbations and Smooth Masks (2019). https://doi.org/10.48550/arXiv.1910.08485, [cs, stat]
Fong, R., Vedaldi, A.: Interpretable explanations of black boxes by meaningful perturbation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3449–3457 (2017). https://doi.org/10.1109/ICCV.2017.371. http://arxiv.org/abs/1704.03296, [cs, stat]
Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-Aware Minimization for Efficiently Improving Generalization (2021). https://doi.org/10.48550/arXiv.2010.01412, [cs, stat]
Gamba, M., Azizpour, H., Björkman, M.: On the Lipschitz Constant of Deep Networks and Double Descent (2023). https://doi.org/10.48550/arXiv.2301.12309, [cs]
Han, T., Srinivas, S., Lakkaraju, H.: Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations (2022). https://doi.org/10.48550/arXiv.2206.01254, [cs]
Harel, N., Gilad-Bachrach, R., Obolski, U.: Inherent Inconsistencies of Feature Importance (2022). https://doi.org/10.48550/arXiv.2206.08204, [cs]
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2015). https://doi.org/10.48550/arXiv.1512.03385, [cs]
Hill, D., Masoomi, A., Ghimire, S., Torop, M., Dy, J.: Explanation Uncertainty with Decision Boundary Awareness (2022). https://doi.org/10.48550/arXiv.2210.02419, [cs]
Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A Benchmark for Interpretability Methods in Deep Neural Networks (2019). https://doi.org/10.48550/arXiv.1806.10758, [cs, stat]
Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007). https://doi.org/10.1109/CVPR.2007.383267. https://ieeexplore.ieee.org/document/4270292. iSSN 1063-6919
Kim, B., Seo, J., Jeon, S., Koo, J., Choe, J., Jeon, T.: Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps (2019). https://doi.org/10.48550/arXiv.1902.04893, [cs, stat]
Kindermans, P.J., et al.: The (Un)reliability of saliency methods (2017). https://doi.org/10.48550/arXiv.1711.00867, [cs, stat]
Kolek, S., Nguyen, D.A., Levie, R., Bruna, J., Kutyniok, G.: Cartoon Explanations of Image Classifiers (2022). https://doi.org/10.48550/arXiv.2110.03485, [cs]
Kolek, S., Windesheim, R., Loarca, H.A., Kutyniok, G., Levie, R.: Explaining Image Classifiers with Multiscale Directional Image Representation (2023). https://doi.org/10.48550/arXiv.2211.12857, [cs]
Krishna, S., et al.: The Disagreement Problem in Explainable Machine Learning: A Practitioner’s Perspective (2022). https://doi.org/10.48550/arXiv.2202.01602, [cs]
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Marmarelis, P.Z., Marmarelis, V.Z.: The white-noise method in system identification. In: Marmarelis, P.Z., Marmarelis, V.Z. (eds.) Analysis of Physiological Systems: The White-Noise Approach, pp. 131–180. Springer, Boston (1978). https://doi.org/10.1007/978-1-4613-3970-0_4
Marques-Silva, J., Ignatiev, A.: Delivering trustworthy AI through formal XAI. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 11, pp. 12342–12350 (2022). https://doi.org/10.1609/aaai.v36i11.21499. https://ojs.aaai.org/index.php/AAAI/article/view/21499
Marx, C., Park, Y., Hasson, H., Wang, Y., Ermon, S., Huan, L.: But are you sure? An uncertainty-aware perspective on explainable AI. In: Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, pp. 7375–7391. PMLR (2023). https://proceedings.mlr.press/v206/marx23a.html
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., Sutskever, I.: Deep Double Descent: Where Bigger Models and More Data Hurt (2019). https://doi.org/10.48550/arXiv.1912.02292, [cs, stat]
Petsiuk, V., Das, A., Saenko, K.: RISE: Randomized Input Sampling for Explanation of Black-box Models (2018). https://doi.org/10.48550/arXiv.1806.07421, [cs]
Rahaman, N., et al.: On the Spectral Bias of Neural Networks (2019). https://doi.org/10.48550/arXiv.1806.08734, [cs, stat]
Rahman, M.: Applications of Fourier Transforms to Generalized Functions. WIT Press (2011). Google-Books-ID: k_rdcKaUdr4C
Ras, G., van Gerven, M., Haselager, P.: Explanation methods in deep learning: users, values, concerns and challenges. In: Escalante, H.J., et al. (eds.) Explainable and Interpretable Models in Computer Vision and Machine Learning. TSSCML, pp. 19–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98131-4_2
Reiter, H., Stegeman, J.D.: Classical Harmonic Analysis and Locally Compact Groups. Oxford University Press (2000). https://doi.org/10.1093/oso/9780198511892.001.0001. https://academic.oup.com/book/54460
Ribeiro, M.T., Singh, S., Guestrin, C.: “Why Should I Trust You?”: Explaining the Predictions of Any Classifier (2016). https://doi.org/10.48550/arXiv.1602.04938, [cs, stat]
Rong, Y., Leemann, T., Borisov, V., Kasneci, G., Kasneci, E.: A Consistent and Efficient Evaluation Strategy for Attribution Methods (2022). https://doi.org/10.48550/arXiv.2202.00449, [cs]
Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206–215 (2019). https://doi.org/10.48550/arXiv.2202.00449, [cs]
Rudin, W.: Principles of Mathematical Analysis. McGraw-Hill (1976). Google-Books-ID: kwqzPAAACAAJ
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2), 336–359 (2020). https://doi.org/10.1007/s11263-019-01228-7. http://arxiv.org/abs/1610.02391, [cs]
Serov, V.: The Riemann–Lebesgue lemma. In: Serov, V. (ed.) Fourier Series, Fourier Transform and Their Applications to Mathematical Physics. AMS, vol. 197, pp. 33–35. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65262-7_6
Shrikumar, A., Greenside, P., Shcherbina, A., Kundaje, A.: Not Just a Black Box: Learning Important Features Through Propagating Activation Differences (2017). https://doi.org/10.48550/arXiv.1605.01713, [cs]
Slack, D., Hilgard, S., Singh, S., Lakkaraju, H.: Reliable Post hoc Explanations: Modeling Uncertainty in Explainability (2021). https://doi.org/10.48550/arXiv.2008.05030, [cs, stat]
Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: SmoothGrad: removing noise by adding noise (2017). https://doi.org/10.48550/arXiv.1706.03825, [cs, stat]
Stoica, P., Moses, R.L.: Spectral Analysis of Signals, vol. 452. Pearson Prentice Hall, Upper Saddle River (2005). http://user.it.uu.se/~ps/SAS-new.pdf
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic Attribution for Deep Networks (2017). https://doi.org/10.48550/arXiv.1703.01365, [cs]
Tancik, M., et al.: Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. arXiv (2020). https://api.semanticscholar.org/CorpusID:219791950
Wang, E., Khosravi, P., Broeck, G.V.: Probabilistic Sufficient Explanations (2021). https://doi.org/10.48550/arXiv.2105.10118, [cs]
Wang, Y., Wang, X.: “Why not other classes?”: towards class-contrastive back-propagation explanations. In: Advances in Neural Information Processing Systems, vol. 35, pp. 9085–9097 (2022). https://proceedings.neurips.cc/paper_files/paper/2022/hash/3b7a66b2d1258e892c89f485b8f896e0-Abstract-Conference.html
Wilming, R., Kieslich, L., Clark, B., Haufe, S.: Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables (2023). https://doi.org/10.48550/arXiv.2306.01464, [cs, stat]
Zeiler, M.D., Fergus, R.: Visualizing and Understanding Convolutional Networks (2013). https://doi.org/10.48550/arXiv.1311.2901, [cs]
Acknowledgements
We would like to thank Giovanni Luca Marchetti for an early review of our work and his kind feedback. This project is partially supported by Region Stockholm through MedTechLabs, and Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Scientific computation was enabled by the supercomputing resource Berzelius, provided by the National Supercomputer Centre at Linköping University and the Knut and Alice Wallenberg foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mehrpanah, A., Englesson, E., Azizpour, H. (2025). On Spectral Properties of Gradient-Based Explanation Methods. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15145. Springer, Cham. https://doi.org/10.1007/978-3-031-73021-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-73021-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73020-7
Online ISBN: 978-3-031-73021-4
eBook Packages: Computer ScienceComputer Science (R0)