Abstract
Predictive process monitoring focuses on forecasting future events in an ongoing sequence of logs coming from a process. Process owners will then use predictions to make decisions or give users information about the advancement of their case. Estimation of uncertainty is a critical uncovered topic in the task of next activity prediction. In this work, we study uncertainty estimation and model calibration in the case of the next activity prediction exploiting deep learning techniques, suggesting a framework able to take advantage of this information in a real scenario. We used an attention-based neural network as a forecasting algorithm and estimated its uncertainty through an ensemble of networks and Monte Carlo dropout. Two public datasets with different complexities and widely adopted in the Process Mining community are used. We investigate the quality of the models’ calibration on the considered datasets and how prediction accuracy is related to uncertainty. We also analyzed test cases in which the model is overconfident and showed that they could be related to uncommon data or sequences not contained in the training set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code available at https://github.com/piepor/uncertain-ppm.git.
References
https://www.mise.gov.it/index.php/it/incentivi/impresa/digital-transformation
Matt, C., et al.: Digital transformation strategies. Bus. Inf. Syst. Eng. 5, 339–343 (2015)
Van der Aalst, W., et al.: Process mining manifesto. In: BPM Workshops (2011)
Bojarski, M., et al.: End to End Learning for Self-Driving Cars. arXiv:1604.07316 (2016)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Camargo, M., Dumas, M., González-Rojas, O.: Learning accurate LSTM models of business processes. In: Hildebrandt, T., van Dongen, B.F., Röglinger, M., Mendling, J. (eds.) BPM 2019. LNCS, vol. 11675, pp. 286–302. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26619-6_19
Zaharah, A., et al.: Process Transformer: Predictive Business Process Monitoring with Transformer Network. arXiv:2104.00721 (2021)
Prasidis, I., et al.: Handling uncertainty in predictive business process monitoring with Bayesian networks. In: International Conference on Information, Intelligence, Systems & Applications (IISA) (2021)
Weytjens, H., De Weerdt, J.: Learning uncertainty with artificial neural networks for improved remaining time prediction of business processes. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 141–157. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_11
Guo, C., et al.: On calibration of modern neural networks. In: ICML (2017)
DeGroot, M.H., Fienberg, S.E.: The comparison and evaluation of forecaster. J. Royal Stat. Soc. Ser. D (The Statistician), 32(1–2), 12–22 (1983)
Bhatt, U., et al.: Uncertainty as a form of transparency: measuring, communicating, and using uncertainty. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (2021)
Gal, Y., Zoubin, G.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: ICML (2016)
Lakshminarayanan, B., et al.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in NIPS (2017)
Hüllermeier, E., Waegeman, W.: Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach. Learn. 110, 457–506 (2021)
Radford, A., et al.: Improving language understanding by generative pre-training. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/languageunsupervised/language understanding paper.pdf (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in NIPS (2017)
Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. Whitepaper (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Verenich, I.: Helpdesk. Mendeley Data (2016)
van Dongen, B.: BPI Challenge 2012. 4TU.ResearchData. Dataset
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Portolani, P., Brusaferri, A., Ballarino, A., Matteucci, M. (2022). Uncertainty in Predictive Process Monitoring. In: Ciucci, D., et al. Information Processing and Management of Uncertainty in Knowledge-Based Systems. IPMU 2022. Communications in Computer and Information Science, vol 1602. Springer, Cham. https://doi.org/10.1007/978-3-031-08974-9_44
Download citation
DOI: https://doi.org/10.1007/978-3-031-08974-9_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08973-2
Online ISBN: 978-3-031-08974-9
eBook Packages: Computer ScienceComputer Science (R0)