Keywords

1 Introduction

Business process models specify in terms of textual or graphical artifacts the business processes in an enterprise [1]. In this context, insights on the comprehension of process models demonstrate that process model comprehension plays an important role when analyzing and optimizing processes [2, 3]. As a result, enterprises are confronted with an influx of process modeling notations (e.g., Business Process Model and Notation (BPMN) 2.0 [4], Event-driven Process Chains (EPCs) [5], or Flow Charts [6]) for the documentation of their business processes within process models. However, for an effective use of process models, the latter must ensure that the processes of an enterprise are comprehended correctly by all involved stakeholders.

In prior research, we investigated process model comprehension in order to reveal factors fostering or thwarting respective comprehension [7, 8]. Furthermore, focusing on cognitive neuroscience and psychology, we proposed valuable lessons learned on how to optimize empirical studies for a deeper investigation on process model comprehension [9].

To enhance our previous work on process model comprehension, this work presents the results obtained from an exploratory process model comprehension study. In detail, a sample consisting of \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts in the domain of process modeling are confronted with process models expressed in terms of eight different process modeling notations. The objective of the study was to evaluate the perceived cognitive load (i.e., effort being used in the working memory) of participants caused when comprehending respective modeling notations. Based on the results we obtained, we derived for each process modeling notation a mental difficulty level.

This work contributes to the field of process model comprehension in two ways. First, in research, we want to learn more about the cognitive load and adverse effects when comprehending process models in terms of different modeling notations [10]. The obtained insights can foster related empirical investigations in this context. Second, in practice, enterprises can be supported in making decision about the adoption of a particular process modeling notation or which modeling tool should be used when adopting process-oriented thinking.

The remainder of the paper is structured as follows: Sect. 2 introduces theoretical backgrounds. Study setting and operation are explained in Sect. 3. In Sect. 4, the obtained results are described empirically and discussed. Finally, Sect. 5 discusses related work, while Sect. 6 summarizes the paper and gives an outlook on future work.

2 Theoretical Background

The cognitive load can be defined as a multidimensional construct representing an individuals cognitive capacity used to work on or to solve a task as well as to address a problem [11]. Thereby, cognitive load has a causal dimension reflecting the interaction between task- (e.g., inherent difficulty of the task) and subject-specific characteristics (e.g., knowledge about a topic). Particularly, cognitive load is comprised of the assessment dimensions describing the measurable aspects mental load and mental effort [12, 13]. The mental load relates to a task, which indicates the cognitive capacity needed to cope with the complexity of a task. Juxtaposing mental load, the mental effort is subject-specific and refers to the invested cognitive capacity of an individual while working on a task [14].

In many fields (e.g., psychology, education), the observation as well as measurement of the cognitive load has become crucial. Reasons for this are that a reduced cognitive load has a positive impact on the working memory, thus promoting information processing and assuring a greater success in learning processes [15]. In turn, a high cognitive load (i.e., overloading the working memory) inhibits information processing leading to confusion and a higher risk of making mistakes. As a consequence, cognitive load should be kept at an appropriate level [16]. Therefore, an appropriate level of cognitive load can be ensured from the ideal interplay of mental load and mental effort. Particularly, by designing tasks and presenting information in such way that an individual is not confronted with challenges, demanding more capacity in the working memory [17].

3 Study Setting

Any process modeling notation has its own strengths and weaknesses regarding, for example, model conformance checking or expressibility [23]. Considering this fact, enterprises are confronted with the important decision about which process modeling notation fulfills their requirements and covers all their needs. Some of the process modeling notations are offering an extensive syntax to express business processes in a fine-grained level. On the other, some notations provide only a limited set, which is, however, sufficient for correctness verification of process models [24]. For an effective use of process modeling notations, their formalisms and methodologies must be comprehended correctly. Thereby, the acquisition of knowledge about a process modeling notation represents a cognitive task. In this context, several aspects of a modeling notation are learned more easily and quickly, while, on the other, some aspects are difficult to learn, having different impact on the cognitive load of an individual. However, this effect cannot be generalized and, hence, is completely different between individuals. Especially novices without any knowledge in process modeling are often confronted with difficulties how to properly comprehend process modeling notations. To address this issue, we conduct an exploratory comprehension study in which novices, intermediates, and experts from the domain Thereby, we want to investigate the impact of modeling notations on the cognitive load (i.e., mental load and mental effort) of individuals. In detail, we agree on using the following notations as described in Table 1. Aside well-known modeling notations (e.g., BPMN), we chose notations that are rarely seen in the process repositories of enterprises (e.g., IDEF3). Table 1 contains an additional column specific (i.e., Spec.), stating whether the use of a modeling notation is more focused on particular aspects.

Table 1. Process modeling notations used in the study

3.1 Study Planning

Participants. All participants have an academic background. In detail, students and research associates as well as professionals, who take a distance e-learning course, are invited for the study at Ulm University. There are no prerequisites for participating in the study and all participants are recruited on a voluntary basis. Further, all participants have given their consent.

Object. Participants need to assess eight different process models (cf. Table 1) regarding their subjective comprehensibility and accessibility. With this assessment, we want to draw conclusions about the perceived mental load and mental effort of the participants. Thereby, the process models reflect three different levels of complexity (i.e., easy, medium, and hard). To be more precise, the easy process model contains only basic modeling elements. With rising level of complexity, the total number of elements is increased and new elements, previously not contained in the process model, are added. For each level of complexity, process models are created using the mentioned eight process modeling notations respectively (cf. Table 1). As the semantic description of the process models is not relevant in this context, we use abstract labels (i.e., alphabetic letters) for the single modeling elements.Footnote 1 Furthermore, it is ensured that all process models are comparable within a specific level of complexity. Therefore, experts and novices in the domain of process modeling, who were not participating in the study, ranked and compared the used process models.

Instrumentation. There are three different questionnaires in the study. First, a pre-study questionnaire is used gathering demographic data (e.g., age, gender) and asking about prior knowledge on process modeling notations. In addition, participants were asked about their familiarity with specific modeling notations.

Second, a mid-study questionnaire is used providing four items regarding the mental load (i.e., comprehensibility and accessibility of the process models): the process model is comprehensible, the process model is accessible, the used modeling constructs (e.g., split and join) are comprehensible, and the the process flow is understood properly. All items are rated on a 7-point Likert scale, ranging from 0 (i.e., strongly disagree) to 6 (i.e., strongly agree).

Third, a post-study questionnaire is used with four items capturing the mental effort of participants: the mental demand for performing the task is high, the task is complex, the overall performance during the task, and the effort level for performing the task. All items are rated on a 7-point Likert scale, ranging from 0 (i.e., strongly disagree) to 6 (i.e., strongly agree).

In addition, in the post-questionnaire participants need to categorize the assessed process models according their subjective preferences regarding the comprehensibility as well as accessibility, beginning from the simplest to the most difficult process model. Finally, participants are able to leave qualitative feedback.

3.2 Study Design and Procedure

The design and procedure of the study is based on the guidelines set out by [25] on how to systematically plan, conduct, and evaluate studies. Precedent, a pilot study with four students and four research associates was conducted for the improvement of the process models and to ensure their comparability with each other. Moreover, the pilot study was used to eliminate potential ambiguities as well as misunderstandings. Further, the chosen language in the study is German. Within a period of two weeks, several sessions for conducting the study are offered to the participants. Each session took about 20 min and ran as follows: An introduction is given to the participants, in which the procedure of the study is explained and the study materials (i.e., process models, questionnaires) are handed out. Afterwards, the participants need to answer the pre-study questionnaire. Following this, they are asked to read and assess eight randomly selected process models. Thereby, it is ensured that participants need to assess each modeling notation in a study run. For each assessed process model, the participants have to answer statements regarding subjective comprehensibility and accessibility (i.e., mental load). After completing this step, participants answer the post-study questionnaire, providing information about perceived mental effort and they need to categorize assessed process models in a ranking system. The study procedure is illustrated in Fig. 1.

Fig. 1.
figure 1

Study design

4 Data Analysis and Interpretation

In total, data from 94 participants were collected. A median split is performed to categorize the participants into samples of novices, intermediates, and experts. Therefore, we determine the median for the number of process models a participant has analyzed and created during the last 12 months. Consequently, \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts participate in the study. Each participant assesses eight randomly selected process models regarding subjective comprehensibility and accessibility (i.e., mental load), resulting in \(n=752\) assessed process models. Table 2 summarizes the detailed distribution of the assessed process modeling notations for each level of complexity.

Table 2. Number of assessed process models

4.1 Descriptive Statistics

The obtained data regarding the perceived mental load for each process modeling notations are presented in Table 3 as means for the entire sample size as well as each sample respectively (i.e., novices, intermediates, and experts). Higher values indicate less mental load. As described in Sect. 3.1, mental load is determined with four aggregated items. As a prerequisite, all response variables must show a high reliability [26]. For this purpose, Cronbach’s \(\alpha \) (i.e., several items are an accurate estimate of an accumulated item) is calculated.Footnote 2 For mental load, a Cronbach with \(\alpha = 0.79 \) was calculated.

Table 3. Mental load

Figures 2, 3 and 4 depict the two-dimensional data (i.e., process modeling notations, level of complexity) regarding the mental load for the entire sample and each sample respectively (i.e., novices, intermediates, and experts) as radar charts. Thereby, higher values stand for less mental load on the used scale.

Fig. 2.
figure 2

Mental load - All

Fig. 3.
figure 3

Mental load - Novices

Fig. 4.
figure 4

Mental load - Intermediates

Fig. 5.
figure 5

Mental load - Experts

Regarding the easy level of complexity (cf. Figs. 2, 3 and 4), several process modeling notations (e.g., BPMN, Petri Nets, and UML Activity Diagrams) are close to the maximum of the scale (i.e., 6 - reflecting less mental effort). However, comparing the values obtained from each sample, there is an indication that easy process models expressed in Declarative or IDEF3 appear to be more challenging to comprehend and, consequently, demand a higher mental load from the participants. In general, with rising level of complexity, an erratic decrease in the mental load is observable (cf. Table 2). In detail, instead of an uniform decrease of the values for the mental load, for several modeling notations, an abrupt decline can be seen in the samples. For example, there is a significant drop in the mental load between the easy and medium BPMN process models in each sample. For novices and intermediates, there is only a little decline in the mental load between the medium and hard BPMN model. However, experts show again a significant drop in the mental load for BPMN process models. Another example concerns the process models expressed in terms of EPCs. There are only slight decreases in the mental load, which indicates that the comprehension of EPC process models appears to be feasible throughout the levels of complexity.

Table 4 shows the mental effort (ME) for the entire sample as well as each sample respectively (i.e., novices, intermediates, and experts). Mental effort is determined by four aggravated items (cf. Sect. 3.1), here, Cronbach resulted in \(\alpha = 0.83\). Higher values stand for less mental effort. The study task demands a moderate mental effort from all samples. Further, there are only minimal differences in the mental effort between novices, intermediates, and experts.

Table 4. Mental effort

4.2 Discussion

It is obvious that with rising level of complexity the mental load regarding the process modeling notations is decreasing (cf. Fig. 2). Despite the comparability of the process models, however, it is interesting to see how the mental load is decreasing in a different manner between novices, intermediates, and experts. For example, the mental load from intermediates regarding the medium and hard process model is about the same. In turn, for novices and experts, the differences between these two levels of complexity are clearly discernible. The same can be observed regarding the IDEF3 process models. Further, the results provide a good indication about the mental load when comprehending process models in particular modeling notations. More complex process models expressed in Declarative, IDEF3, or eGantt appear to be more challenging to comprehend. Reason might be that these notations are not as widespread in practice as other notations. Often, amongst others, tertiary educational institutions are teaching more common notations widely used in practice such as BPMN and EPCs.

While there are different characteristics for the mental load, however, the mental effort needed for process model comprehension is approximately the same between novices, intermediates, and experts. Although the mental effort for comprehending a process model is on a moderate level, however, process model comprehension is a complex matter that needs to be taken into account.

Based on the study results, Fig. 6 presents the derived mental difficulty for each process modeling notation and respective level of complexity. Therefore, for each process model, the percentage proportion is calculated based on the categorization the participants indicated in the post-study questionnaire (cf. Sect. 3.1). Therefore, we considered respective position of each process model as well as their level of complexity. The calculated percentage proportion serves as a factor used for multiplying the aggravated constructs mental load and mental effort to derive the mental difficulty [27].

Fig. 6.
figure 6

Mental difficulty

As shown in Fig. 6, regarding an easy level of complexity, almost all values, except for Declarative and IDEF3, are still in the green range. This means that the comprehension of such process models should not be a challenge. With rising level of complexity, the values for the comprehension of process models tend towards the orange or red range respectively. However, none of the values is located completely in the red range and, consequently, it seems to be that process models can be comprehended independently of the modeling notation and level of complexity. In addition, the results confirm the further use of the widespread and established modeling notations such as BPMN and EPCs.

4.3 Threats to Validity

Study results can only be accurately interpreted when limitations are also discussed. First, the comparison of the different process modeling notations can also be seen as a comparison like the idiom apples and oranges. Particular modeling notations are usually not used for the direct modeling of business processes, but for other purposes (e.g., Petri Nets are mainly used for the verification of process models) and, hence, a direct comparison is difficult. Second, the classification of participants into the samples of novices, intermediates, and experts solely based on a median split by considering the number of process models a participant has analyzed and created during the last 12 months may be oversimplified. Hence, results might differ significantly with experts in process modeling with several years of experience. Third, the size of the process models might not be representative. In general, real world process models are usually much larger than the models we used in the study. Fourth, in addition, to ensure comparability between the process models, the latter are modeled only with basic modeling elements because some modeling notations don’t have the expressiveness for the documentation of complex processes. Fifth, the respective level of complexity reflected by the process models constitutes another threat. The models might be considerably unbalanced between the level of complexity and, hence, working memory capacity of participants, especially for novices, may be exceeded. Seventh, specific process modeling notations (e.g., BPMN) may be considered as easy to comprehend since these notations are more familiar than others (e.g., IDEF3), thus resulting in a positive impact on mental load and mental effort.

5 Related Work

A review of existing business process modeling notations in literature is presented in [23]. Further, the authors present a framework for the classification of modeling notations according their purpose. [28] discusses an evaluation of modeling notations and proposes a meta-model to capture the concepts of the evaluated notations. In turn, [29] compares and discusses different process modeling methods based on aspects like process model representation and tool support.

Concerning research on process model comprehension, [30] gives insights into subject-specific characteristics (e.g., theoretical knowledge) influencing process model comprehension. In turn, [31] evaluates different process modeling notations with respect to their comprehensibility. A discussion about factors having an influence on the comprehension of process models is presented in [32].

Regarding cognitive aspects, a measure to determine the cognitive complexity for process models is proposed in [33]. Further, [34] shows an approach to reduce the cognitive load when comprehending process models by applying patterns for improving the model comprehensibility. The empirical assessment of participants’ mental effort, while creating or comprehending process models, is demonstrated in [35]. The mental effort needed between inexperienced and experienced modelers to create process models is investigated in [36].

Altogether, there are several works regarding the comparison of process modeling notations. However, to the best of our knowledge, none of the discussed works deal with such a comparison of process modeling notations, while taking the cognitive load (i.e., mental load and mental effort) of participants into account.

6 Summary and Outlook

This paper presented the impact of different process modeling notations on the cognitive load (i.e., mental load and mental effort) of \(n=38\) novices, \(n=21\) intermediates, and \(n=35\) experts in the domain of process modeling. Therefore, study participants needed to comprehend and assess process models of different levels of complexity (i.e., easy, medium, hard) in terms of eight different modeling notations, i.e., BPMN 2.0, Declarative Process Modeling, eGantt Charts, EPCs, Flow Charts, IDEF3, Petri Nets, and UML Activity Diagrams. We gathered information related to the mental effort and mental load (i.e., cognitive load) of participants while performing the study task. Based on the cognitive load, we derived a factor representing the mental difficulty for each process modeling notation. The high availability of process modeling notations resulted in a strong demand from enterprises to compare and evaluate these notations in order to find an appropriate one covering the needs of an enterprise. The presented mental difficulty may support enterprises in making this decision considering the perceived cognitive load of an individual, while comprehending process models expressed in terms of different modeling notations. In addition, the results from this paper may help in answering questions about which modeling notations should be supported with a greater emphasis in modeling tools. Further, the results show that the use of established modeling notations (e.g., BPMN) is recommendable for enterprises and future process modelers as well as analysts.

Future research is needed in order to determine more precise indications about an individuals cognitive load while comprehending process models. Besides an in-depth statistical analysis of the results, for example, instead of using abstract labels the process model elements could be described with concrete labels. Further, participants should be confronted with process models from real projects expressed in terms of the assessed modeling notations. Finally, cognitive load is only one factor having an effect on the comprehension of process models expressed in different modeling notations and, therefore, the consideration of additional factors (e.g., expressiveness, level of automation) will be subject of future studies.