Abstract
Explanation for Multivariate Time Series Classification (MTSC) is an important topic that is under explored. There are very few quantitative evaluation methodologies and even fewer examples of actionable explanation, where the explanation methods are shown to objectively improve specific computational tasks on time series data. In this paper we focus on analyzing InterpretTime, a recent evaluation methodology for attribution methods applied to MTSC. We showcase some significant weaknesses of the original methodology and propose ideas to improve both its accuracy and efficiency. Unlike related work, we go beyond evaluation and also showcase the actionability of the produced explainer ranking, by using the best attribution methods for the task of channel selection in MTSC. We find that perturbation-based methods such as SHAP and Feature Ablation work well across a set of datasets, classifiers and tasks and outperform gradient-based methods. We apply the best ranked explainers to channel selection for MTSC and show significant data size reduction and improved classifier accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Boniol, P., Meftah, M., Remy, E., Palpanas, T.: DCAM: dimension-wise class activation map for explaining multivariate data series classification. In: SIGMOD (2022)
Clark, K., Khandelwal, U., Levy, O., Manning, C.D.: What does BERT look at? An analysis of BERT’s attention. In: ACL (2019)
Dau, H.A., et al.: The UCR time series archive. IEEE/CAA J. Automatica Sinica (2019)
Dempster, A., Petitjean, F., Webb, G.I.: Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. DAMI (2019)
Dempster, A., Schmidt, D.F., Webb, G.I.: Minirocket: a very fast (almost) deterministic transform for time series classification. In: SIGKDD (2021)
Der, A., et al.: Pupae: intuitive and actionable explanations for time series anomalies. In: SDM (2024)
Dhariyal, B., Le Nguyen, T., Ifrim, G.: Scalable classifier-agnostic channel selection for multivariate time series classification. DAMI (2023)
Dhariyal, B., Nguyen, T.L., Ifrim, G.: Fast channel selection for scalable multivariate time series classification. In: AALTD (2021)
Etmann, C., Lunz, S., Maass, P., Schönlieb, C.B.: On the connection between adversarial robustness and saliency map interpretability. In: ICML (2019)
Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many are useful: learning a variable’s importance by studying an entire class of prediction models simultaneously. JMLR (2019)
Foumani, N.M., Tan, C.W., Webb, G.I., Salehi, M.: Improving position encoding of transformers for multivariate time series classification. DAMI (2023)
Gomez, T., Fréour, T., Mouchère, H.: Metrics for saliency map evaluation of deep learning explanation methods. In: ICPRAI (2022)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. (2019)
Ismail Fawaz, H., et al.: Inceptiontime: finding alexnet for time series classification. DAMI (2020)
Kokhlikyan, N., et al.: Captum: a unified and generic model interpretability library for PyTorch. In: KDD (2020)
Le Nguyen, T., Gsponer, S., Ilie, I., O’reilly, M., Ifrim, G.: Interpretable time series classification using linear models and multi-resolution multi-domain symbolic representations. DAMI (2019)
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: NeurIPS (2017)
Mao, K., Dou, Z., Qian, H., Mo, F., Cheng, X., Cao, Z.: Convtrans: transforming web search sessions for conversational dense retrieval. In: EMNLP (2022)
Nguyen, T.T., Nguyen, T.L., Ifrim, G.: AMEE: a robust framework for explanation evaluation in time series classification. arXiv preprint arXiv:2306.05501 (2023)
Nguyen, T.T., Nguyen, T.L., Ifrim, G.: Robust explainer recommendation for time series classification. Data Min. Knowl. Discov. (2024)
Oguiza, I.: TSAI - a state-of-the-art deep learning library for time series and sequential data. Github (2023). https://github.com/timeseriesAI/tsai
Pasos-Ruiz, A., Flynn, M., Bagnall, A.: Benchmarking multivariate time series classification algorithms. DAMI (2020)
Serramazza, D.I., Nguyen, T.T., Le Nguyen, T., Ifrim, G.: Evaluating explanation methods for multivariate time series classification. In: AALTD (2023)
Singh, A., et al.: Fast and robust video-based exercise classification via body pose tracking and scalable multivariate time series classifiers. DAMI (2023)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML (2017)
Tan, C.W., Dempster, A., Bergmeir, C., Webb, G.I.: Multirocket: multiple pooling operators and transformations for fast and effective time series classification. DAMI (2022)
Theissler, A., Spinnato, F., Schlegel, U., Guidotti, R.: Explainable AI for time series classification: a review, taxonomy and research directions. IEEE Access 10, 100700–100724 (2022)
Turbé, H., Bjelogrlic, M., Lovis, C., Mengaldo, G.: Evaluation of post-hoc interpretability methods in time-series classification. Nat. Mach. Intell. (2023)
Zhou, B., Khosla, A., A., L., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR (2016)
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: PMLR (2017)
Assaf, R., Giurgiu, I., Bagehorn, F., Schumann, A.: Mtex-CNN: multivariate time series explanations for predictions with convolutional neural networks. In: 2019 IEEE International Conference on Data Mining (ICDM), pp. 952–957. IEEE (2019)
Bi, X., Zhang, C., He, Y., Zhao, X., Sun, Y., Ma, Y.: Explainable time-frequency convolutional neural network for microseismic waveform classification. Inf. Sci. 546, 883–896 (2021)
Maweu, B.M., Dakshit, S., Shamsuddin, R., Prabhakaran, B.: CEFEs: a CNN explainable framework for ECG signals. Artif. Intell. Med. 115, 102059 (2021)
Munir, M., Siddiqui, S.A., Küsters, F., Mercier, D., Dengel, A., Ahmed, S.: TSXplain: demystification of DNN decisions for time-series using natural language and statistical features. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11731, pp. 426–439. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30493-5_43
Siddiqui, S.A., Mercier, D., Munir, M., Dengel, A., Ahmed, S.: Tsviz: demystification of deep learning models for time-series analysis. IEEE Access 7, 67027–67040 (2019)
Sivill, T., Flach, P.: Limesegment: meaningful, realistic time series explanations. In: International Conference on Artificial Intelligence and Statistics, pp. 3418–3433. PMLR (2022)
Acknowledgments
We thank the anonymous reviewers for their constructive feedback. We are grateful to Jiawen Wei and Gianmarco Mengaldo for detailed discussions on the original InterpretTime methodology. We thank all researchers working on time series and explainable AI who have made their data, code and results open source to help the reproducibility of research methods in this area. This work was funded by Science Foundation Ireland through the SFI Centre for Research Training in Machine Learning (18/CRT/6183) and the Insight Centre for Data Analytics (12/RC/2289_P2). For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Serramazza, D.I., Nguyen, T.L., Ifrim, G. (2024). Improving the Evaluation and Actionability of Explanation Methods for Multivariate Time Series Classification. In: Bifet, A., Davis, J., Krilavičius, T., Kull, M., Ntoutsi, E., Žliobaitė, I. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2024. Lecture Notes in Computer Science(), vol 14944. Springer, Cham. https://doi.org/10.1007/978-3-031-70359-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-70359-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70358-4
Online ISBN: 978-3-031-70359-1
eBook Packages: Computer ScienceComputer Science (R0)