Abstract
Over the last decade, plenty business process modeling notations emerged for the documentation of business processes in enterprises. During the learning of a modeling notation, an individual is confronted with a cognitive load that has an impact on the comprehension of a notation with its underlying formalisms and concepts. To address the cognitive load, this paper presents the results from an exploratory study, in which a sample of 94 participants, divided into novices, intermediates, and experts, needed to assess process models expressed in terms of eight different process modeling notations, i.e., BPMN 2.0, Declarative Process Modeling, eGantt Charts, EPCs, Flow Charts, IDEF3, Petri Nets, and UML Activity Diagrams. The study focus was set on the subjective comprehensibility and accessibility of process models reflecting participant’s cognitive load (i.e., mental load and mental effort). Based on the cognitive load, a factor reflecting the mental difficulty for comprehending process models in different modeling notations was derived. The results indicate that established modeling notations from industry (e.g., BPMN) should be the first choice for enterprises when striving for process management. Moreover, study insights may be used to determine which modeling notations should be taught for an introduction in process modeling or which notation is useful to teach and train process modelers or analysts.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Business process models specify in terms of textual or graphical artifacts the business processes in an enterprise [1]. In this context, insights on the comprehension of process models demonstrate that process model comprehension plays an important role when analyzing and optimizing processes [2, 3]. As a result, enterprises are confronted with an influx of process modeling notations (e.g., Business Process Model and Notation (BPMN) 2.0 [4], Event-driven Process Chains (EPCs) [5], or Flow Charts [6]) for the documentation of their business processes within process models. However, for an effective use of process models, the latter must ensure that the processes of an enterprise are comprehended correctly by all involved stakeholders.
In prior research, we investigated process model comprehension in order to reveal factors fostering or thwarting respective comprehension [7, 8]. Furthermore, focusing on cognitive neuroscience and psychology, we proposed valuable lessons learned on how to optimize empirical studies for a deeper investigation on process model comprehension [9].
To enhance our previous work on process model comprehension, this work presents the results obtained from an exploratory process model comprehension study. In detail, a sample consisting of \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts in the domain of process modeling are confronted with process models expressed in terms of eight different process modeling notations. The objective of the study was to evaluate the perceived cognitive load (i.e., effort being used in the working memory) of participants caused when comprehending respective modeling notations. Based on the results we obtained, we derived for each process modeling notation a mental difficulty level.
This work contributes to the field of process model comprehension in two ways. First, in research, we want to learn more about the cognitive load and adverse effects when comprehending process models in terms of different modeling notations [10]. The obtained insights can foster related empirical investigations in this context. Second, in practice, enterprises can be supported in making decision about the adoption of a particular process modeling notation or which modeling tool should be used when adopting process-oriented thinking.
The remainder of the paper is structured as follows: Sect. 2 introduces theoretical backgrounds. Study setting and operation are explained in Sect. 3. In Sect. 4, the obtained results are described empirically and discussed. Finally, Sect. 5 discusses related work, while Sect. 6 summarizes the paper and gives an outlook on future work.
2 Theoretical Background
The cognitive load can be defined as a multidimensional construct representing an individuals cognitive capacity used to work on or to solve a task as well as to address a problem [11]. Thereby, cognitive load has a causal dimension reflecting the interaction between task- (e.g., inherent difficulty of the task) and subject-specific characteristics (e.g., knowledge about a topic). Particularly, cognitive load is comprised of the assessment dimensions describing the measurable aspects mental load and mental effort [12, 13]. The mental load relates to a task, which indicates the cognitive capacity needed to cope with the complexity of a task. Juxtaposing mental load, the mental effort is subject-specific and refers to the invested cognitive capacity of an individual while working on a task [14].
In many fields (e.g., psychology, education), the observation as well as measurement of the cognitive load has become crucial. Reasons for this are that a reduced cognitive load has a positive impact on the working memory, thus promoting information processing and assuring a greater success in learning processes [15]. In turn, a high cognitive load (i.e., overloading the working memory) inhibits information processing leading to confusion and a higher risk of making mistakes. As a consequence, cognitive load should be kept at an appropriate level [16]. Therefore, an appropriate level of cognitive load can be ensured from the ideal interplay of mental load and mental effort. Particularly, by designing tasks and presenting information in such way that an individual is not confronted with challenges, demanding more capacity in the working memory [17].
3 Study Setting
Any process modeling notation has its own strengths and weaknesses regarding, for example, model conformance checking or expressibility [23]. Considering this fact, enterprises are confronted with the important decision about which process modeling notation fulfills their requirements and covers all their needs. Some of the process modeling notations are offering an extensive syntax to express business processes in a fine-grained level. On the other, some notations provide only a limited set, which is, however, sufficient for correctness verification of process models [24]. For an effective use of process modeling notations, their formalisms and methodologies must be comprehended correctly. Thereby, the acquisition of knowledge about a process modeling notation represents a cognitive task. In this context, several aspects of a modeling notation are learned more easily and quickly, while, on the other, some aspects are difficult to learn, having different impact on the cognitive load of an individual. However, this effect cannot be generalized and, hence, is completely different between individuals. Especially novices without any knowledge in process modeling are often confronted with difficulties how to properly comprehend process modeling notations. To address this issue, we conduct an exploratory comprehension study in which novices, intermediates, and experts from the domain Thereby, we want to investigate the impact of modeling notations on the cognitive load (i.e., mental load and mental effort) of individuals. In detail, we agree on using the following notations as described in Table 1. Aside well-known modeling notations (e.g., BPMN), we chose notations that are rarely seen in the process repositories of enterprises (e.g., IDEF3). Table 1 contains an additional column specific (i.e., Spec.), stating whether the use of a modeling notation is more focused on particular aspects.
3.1 Study Planning
Participants. All participants have an academic background. In detail, students and research associates as well as professionals, who take a distance e-learning course, are invited for the study at Ulm University. There are no prerequisites for participating in the study and all participants are recruited on a voluntary basis. Further, all participants have given their consent.
Object. Participants need to assess eight different process models (cf. Table 1) regarding their subjective comprehensibility and accessibility. With this assessment, we want to draw conclusions about the perceived mental load and mental effort of the participants. Thereby, the process models reflect three different levels of complexity (i.e., easy, medium, and hard). To be more precise, the easy process model contains only basic modeling elements. With rising level of complexity, the total number of elements is increased and new elements, previously not contained in the process model, are added. For each level of complexity, process models are created using the mentioned eight process modeling notations respectively (cf. Table 1). As the semantic description of the process models is not relevant in this context, we use abstract labels (i.e., alphabetic letters) for the single modeling elements.Footnote 1 Furthermore, it is ensured that all process models are comparable within a specific level of complexity. Therefore, experts and novices in the domain of process modeling, who were not participating in the study, ranked and compared the used process models.
Instrumentation. There are three different questionnaires in the study. First, a pre-study questionnaire is used gathering demographic data (e.g., age, gender) and asking about prior knowledge on process modeling notations. In addition, participants were asked about their familiarity with specific modeling notations.
Second, a mid-study questionnaire is used providing four items regarding the mental load (i.e., comprehensibility and accessibility of the process models): the process model is comprehensible, the process model is accessible, the used modeling constructs (e.g., split and join) are comprehensible, and the the process flow is understood properly. All items are rated on a 7-point Likert scale, ranging from 0 (i.e., strongly disagree) to 6 (i.e., strongly agree).
Third, a post-study questionnaire is used with four items capturing the mental effort of participants: the mental demand for performing the task is high, the task is complex, the overall performance during the task, and the effort level for performing the task. All items are rated on a 7-point Likert scale, ranging from 0 (i.e., strongly disagree) to 6 (i.e., strongly agree).
In addition, in the post-questionnaire participants need to categorize the assessed process models according their subjective preferences regarding the comprehensibility as well as accessibility, beginning from the simplest to the most difficult process model. Finally, participants are able to leave qualitative feedback.
3.2 Study Design and Procedure
The design and procedure of the study is based on the guidelines set out by [25] on how to systematically plan, conduct, and evaluate studies. Precedent, a pilot study with four students and four research associates was conducted for the improvement of the process models and to ensure their comparability with each other. Moreover, the pilot study was used to eliminate potential ambiguities as well as misunderstandings. Further, the chosen language in the study is German. Within a period of two weeks, several sessions for conducting the study are offered to the participants. Each session took about 20 min and ran as follows: An introduction is given to the participants, in which the procedure of the study is explained and the study materials (i.e., process models, questionnaires) are handed out. Afterwards, the participants need to answer the pre-study questionnaire. Following this, they are asked to read and assess eight randomly selected process models. Thereby, it is ensured that participants need to assess each modeling notation in a study run. For each assessed process model, the participants have to answer statements regarding subjective comprehensibility and accessibility (i.e., mental load). After completing this step, participants answer the post-study questionnaire, providing information about perceived mental effort and they need to categorize assessed process models in a ranking system. The study procedure is illustrated in Fig. 1.
4 Data Analysis and Interpretation
In total, data from 94 participants were collected. A median split is performed to categorize the participants into samples of novices, intermediates, and experts. Therefore, we determine the median for the number of process models a participant has analyzed and created during the last 12 months. Consequently, \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts participate in the study. Each participant assesses eight randomly selected process models regarding subjective comprehensibility and accessibility (i.e., mental load), resulting in \(n=752\) assessed process models. Table 2 summarizes the detailed distribution of the assessed process modeling notations for each level of complexity.
4.1 Descriptive Statistics
The obtained data regarding the perceived mental load for each process modeling notations are presented in Table 3 as means for the entire sample size as well as each sample respectively (i.e., novices, intermediates, and experts). Higher values indicate less mental load. As described in Sect. 3.1, mental load is determined with four aggregated items. As a prerequisite, all response variables must show a high reliability [26]. For this purpose, Cronbach’s \(\alpha \) (i.e., several items are an accurate estimate of an accumulated item) is calculated.Footnote 2 For mental load, a Cronbach with \(\alpha = 0.79 \) was calculated.
Figures 2, 3 and 4 depict the two-dimensional data (i.e., process modeling notations, level of complexity) regarding the mental load for the entire sample and each sample respectively (i.e., novices, intermediates, and experts) as radar charts. Thereby, higher values stand for less mental load on the used scale.
Regarding the easy level of complexity (cf. Figs. 2, 3 and 4), several process modeling notations (e.g., BPMN, Petri Nets, and UML Activity Diagrams) are close to the maximum of the scale (i.e., 6 - reflecting less mental effort). However, comparing the values obtained from each sample, there is an indication that easy process models expressed in Declarative or IDEF3 appear to be more challenging to comprehend and, consequently, demand a higher mental load from the participants. In general, with rising level of complexity, an erratic decrease in the mental load is observable (cf. Table 2). In detail, instead of an uniform decrease of the values for the mental load, for several modeling notations, an abrupt decline can be seen in the samples. For example, there is a significant drop in the mental load between the easy and medium BPMN process models in each sample. For novices and intermediates, there is only a little decline in the mental load between the medium and hard BPMN model. However, experts show again a significant drop in the mental load for BPMN process models. Another example concerns the process models expressed in terms of EPCs. There are only slight decreases in the mental load, which indicates that the comprehension of EPC process models appears to be feasible throughout the levels of complexity.
Table 4 shows the mental effort (ME) for the entire sample as well as each sample respectively (i.e., novices, intermediates, and experts). Mental effort is determined by four aggravated items (cf. Sect. 3.1), here, Cronbach resulted in \(\alpha = 0.83\). Higher values stand for less mental effort. The study task demands a moderate mental effort from all samples. Further, there are only minimal differences in the mental effort between novices, intermediates, and experts.
4.2 Discussion
It is obvious that with rising level of complexity the mental load regarding the process modeling notations is decreasing (cf. Fig. 2). Despite the comparability of the process models, however, it is interesting to see how the mental load is decreasing in a different manner between novices, intermediates, and experts. For example, the mental load from intermediates regarding the medium and hard process model is about the same. In turn, for novices and experts, the differences between these two levels of complexity are clearly discernible. The same can be observed regarding the IDEF3 process models. Further, the results provide a good indication about the mental load when comprehending process models in particular modeling notations. More complex process models expressed in Declarative, IDEF3, or eGantt appear to be more challenging to comprehend. Reason might be that these notations are not as widespread in practice as other notations. Often, amongst others, tertiary educational institutions are teaching more common notations widely used in practice such as BPMN and EPCs.
While there are different characteristics for the mental load, however, the mental effort needed for process model comprehension is approximately the same between novices, intermediates, and experts. Although the mental effort for comprehending a process model is on a moderate level, however, process model comprehension is a complex matter that needs to be taken into account.
Based on the study results, Fig. 6 presents the derived mental difficulty for each process modeling notation and respective level of complexity. Therefore, for each process model, the percentage proportion is calculated based on the categorization the participants indicated in the post-study questionnaire (cf. Sect. 3.1). Therefore, we considered respective position of each process model as well as their level of complexity. The calculated percentage proportion serves as a factor used for multiplying the aggravated constructs mental load and mental effort to derive the mental difficulty [27].
As shown in Fig. 6, regarding an easy level of complexity, almost all values, except for Declarative and IDEF3, are still in the green range. This means that the comprehension of such process models should not be a challenge. With rising level of complexity, the values for the comprehension of process models tend towards the orange or red range respectively. However, none of the values is located completely in the red range and, consequently, it seems to be that process models can be comprehended independently of the modeling notation and level of complexity. In addition, the results confirm the further use of the widespread and established modeling notations such as BPMN and EPCs.
4.3 Threats to Validity
Study results can only be accurately interpreted when limitations are also discussed. First, the comparison of the different process modeling notations can also be seen as a comparison like the idiom apples and oranges. Particular modeling notations are usually not used for the direct modeling of business processes, but for other purposes (e.g., Petri Nets are mainly used for the verification of process models) and, hence, a direct comparison is difficult. Second, the classification of participants into the samples of novices, intermediates, and experts solely based on a median split by considering the number of process models a participant has analyzed and created during the last 12 months may be oversimplified. Hence, results might differ significantly with experts in process modeling with several years of experience. Third, the size of the process models might not be representative. In general, real world process models are usually much larger than the models we used in the study. Fourth, in addition, to ensure comparability between the process models, the latter are modeled only with basic modeling elements because some modeling notations don’t have the expressiveness for the documentation of complex processes. Fifth, the respective level of complexity reflected by the process models constitutes another threat. The models might be considerably unbalanced between the level of complexity and, hence, working memory capacity of participants, especially for novices, may be exceeded. Seventh, specific process modeling notations (e.g., BPMN) may be considered as easy to comprehend since these notations are more familiar than others (e.g., IDEF3), thus resulting in a positive impact on mental load and mental effort.
5 Related Work
A review of existing business process modeling notations in literature is presented in [23]. Further, the authors present a framework for the classification of modeling notations according their purpose. [28] discusses an evaluation of modeling notations and proposes a meta-model to capture the concepts of the evaluated notations. In turn, [29] compares and discusses different process modeling methods based on aspects like process model representation and tool support.
Concerning research on process model comprehension, [30] gives insights into subject-specific characteristics (e.g., theoretical knowledge) influencing process model comprehension. In turn, [31] evaluates different process modeling notations with respect to their comprehensibility. A discussion about factors having an influence on the comprehension of process models is presented in [32].
Regarding cognitive aspects, a measure to determine the cognitive complexity for process models is proposed in [33]. Further, [34] shows an approach to reduce the cognitive load when comprehending process models by applying patterns for improving the model comprehensibility. The empirical assessment of participants’ mental effort, while creating or comprehending process models, is demonstrated in [35]. The mental effort needed between inexperienced and experienced modelers to create process models is investigated in [36].
Altogether, there are several works regarding the comparison of process modeling notations. However, to the best of our knowledge, none of the discussed works deal with such a comparison of process modeling notations, while taking the cognitive load (i.e., mental load and mental effort) of participants into account.
6 Summary and Outlook
This paper presented the impact of different process modeling notations on the cognitive load (i.e., mental load and mental effort) of \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts in the domain of process modeling. Therefore, study participants needed to comprehend and assess process models of different levels of complexity (i.e., easy, medium, hard) in terms of eight different modeling notations, i.e., BPMN 2.0, Declarative Process Modeling, eGantt Charts, EPCs, Flow Charts, IDEF3, Petri Nets, and UML Activity Diagrams. We gathered information related to the mental effort and mental load (i.e., cognitive load) of participants while performing the study task. Based on the cognitive load, we derived a factor representing the mental difficulty for each process modeling notation. The high availability of process modeling notations resulted in a strong demand from enterprises to compare and evaluate these notations in order to find an appropriate one covering the needs of an enterprise. The presented mental difficulty may support enterprises in making this decision considering the perceived cognitive load of an individual, while comprehending process models expressed in terms of different modeling notations. In addition, the results from this paper may help in answering questions about which modeling notations should be supported with a greater emphasis in modeling tools. Further, the results show that the use of established modeling notations (e.g., BPMN) is recommendable for enterprises and future process modelers as well as analysts.
Future research is needed in order to determine more precise indications about an individuals cognitive load while comprehending process models. Besides an in-depth statistical analysis of the results, for example, instead of using abstract labels the process model elements could be described with concrete labels. Further, participants should be confronted with process models from real projects expressed in terms of the assessed modeling notations. Finally, cognitive load is only one factor having an effect on the comprehension of process models expressed in different modeling notations and, therefore, the consideration of additional factors (e.g., expressiveness, level of automation) will be subject of future studies.
Notes
- 1.
- 2.
According to [26], \(\alpha>\) 0.6 acceptable reliability; 0.7 \(<\alpha<\) 0.9 good reliability.
References
Nowak, A., et al.: Flexible information design for business process visualizations. In: 5th International Conference on Service-Oriented Comp and App (SOCA 2012), pp. 1–8 (2012)
Bandara, W., Indulska, M., Chong, S., Sadiq, S.: Major issues in business process management: An Expert Perspective, pp. 1240–1251 (2007)
Gulla, J.A., Brasethvik, T.: On the challenges of business modeling in large-scale reengineering projects. In: International Conference on Requirements Engineering, pp. 17–26 (2000)
OMG: Business Process Management & Notation 2.0 (2018). www.bpmn.org. Accessed 27 Feb 2018
van der Aalst, W.M.P.: Formalization and verification of event-driven process chains. Inf. Soft, Tech., 41(10), 639–650 (1999)
Schultheiss, L.A., Heiliger, E.: Techniques of flow-charting. In: Proceedings of the 1963 Clinic on Library Applications of Data Processeing, pp. 62–78 (1963)
Zimoch, M., et al.: Cognitive insights into business process model comprehension: preliminary results for experienced and inexperienced individuals. In: Proceedings of the BPMDS 2017, pp. 137–152 (2017)
Zimoch, M., et al.: Using insights from cognitive neuroscience to investigate the effects of event-driven process chains on process model comprehension. In: Proceedings of the 1st International Conference on Cognitive Business Process Management, pp. 446–459 (2017)
Zimoch, M., et al.: Eye tracking experiments on process model comprehension: lessons learned. In: Proceedings of the BPMDS 2017, pp. 153–168 (2017)
Zugal, S., Pinggera, J., Weber, B.: Assessing process models with cognitive psychology. EMISA 190, 177–182 (2011)
Paas, F., et al.: Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38, 63–71 (2003)
Wickens, C.D.: Multiple resources and mental workload. Hum. Factors 50, 449–455 (2008)
Paas, F.G., Van Merriënboer, J.J.: The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Hum. Factors 35, 737–743 (1993)
Krell, M.: Evaluating an instrument to measure mental load and mental effort considering different sources of validity evidence. Cogent Educ. 4, 1280256 (2017)
Sweller, J.: Cognitive load during problem solving: effects on learning. Cogn. Sci. 12, 257–285 (1988)
Ayres, P.: Impact of reducing intrinsic cognitive load on learning in a mathematical domain. Appl. Cogn. Psychol. 20, 287–298 (2006)
Baddeley, A.: Working memory. Science 255, 556–559 (1992)
van Der Aalst, W.M., et al.: Declarative workflows: balancing between flexibility and support. Comp. Sci-Res. Dev. 23, 99–113 (2009)
Lanz, A., Kolb, J., Reichert, M.: Enabling personalized process schedules with time-aware process views. In: Franch, X., Soffer, P. (eds.) CAiSE 2013. LNBIP, vol. 148, pp. 205–216. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38490-5_20
Kim, C.H., Yim, D.S., Weston, R.: An integrated use of IDEFO, IDEF3 and petri net methods in support of business process modelling. Proc. Inst. Mech. Eng. Part E: J. Process. Mech. Eng. 215(4), 317–329 (2001)
Murata, T.: Petri Nets: properties, analysis and applications. Proc. IEEE 77, 541–580 (1989)
Dumas, M., ter Hofstede, A.H.M.: UML activity diagrams as a workflow specification language. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 76–90. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45441-1_7
Aguilar-Saven, R.S.: Business process modelling: review and framework. Int. J. Prod. Econ. 90, 129–149 (2004)
White, S.A.: Process modeling notations and workflow patterns. In: Workflow Handbook, pp. 265–294 (2004)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslen, A.: Experimentation in Software Engineering - An Introduction. Kluwer, New York (2000)
Kline, P.: Handbook of Psychological Testing, vol. 2. Routledge, Abingdon (1999)
Sweller, J.: Cognitive load theory, learning difficulty, and instructional design. Learn. Instr. 4, 295–312 (1994)
List, B., Korherr, B.: An evaluation of conceptual business process modelling languages. In: Proceedings of the Symposium on Applied Computing (SAC 2006), pp. 1532–1539 (2006)
Wang, W., Ding, H., Dong, J., Ren, C.: A comparison of business process modeling methods. In: International Conference on SOLI 2006, pp. 1136–1141 (2006)
Mendling, J., Recker, J., Reijers, H.A., Leopold, H.: An empirical review of the connection between model viewer characteristics and the comprehension of conceptual process models. In: Information Systems Frontiers, pp. 1–25 (2018)
Kiepuszewski, B., ter Hofstede, A.H.M., Bussler, C.J.: On structured workflow modelling. In: Wangler, B., Bergman, L. (eds.) CAiSE 2000. LNCS, vol. 1789, pp. 431–445. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45140-4_29
Mendling, J., Strembeck, M., Recker, J.: Factors of process model comprehension-findings from a series of experiments. Decis. Support Syst. 53(1), 195–206 (2012)
Gruhn, V., Laue, R.: Adopting the cognitive complexity measure for business process models. In: 5th International Conference on Cognitive Informatics (ICCI 2006), pp. 236–241 (2006)
Gruhn, V., Laue, R.: Reducing the cognitive complexity of business process models. In: 8th International Conference on Cognitive Informatics (ICCI 2009), pp. 339–345 (2009)
Zugal, S., Pinggera, J., Reijers, H., Reichert, M., Weber, B.: Making the case for measuring mental effort. In: EESSMod 2012 (2012)
Martini, M., et al.: the impact of working memory and the process of process modelling on model quality: investigating experienced versus inexperienced modellers. In: Scientific Reports, vol. 6 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zimoch, M., Pryss, R., Probst, T., Schlee, W., Reichert, M. (2019). The Repercussions of Business Process Modeling Notations on Mental Load and Mental Effort. In: Daniel, F., Sheng, Q., Motahari, H. (eds) Business Process Management Workshops. BPM 2018. Lecture Notes in Business Information Processing, vol 342. Springer, Cham. https://doi.org/10.1007/978-3-030-11641-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-11641-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11640-8
Online ISBN: 978-3-030-11641-5
eBook Packages: Computer ScienceComputer Science (R0)