Abstract
The increasing demand for software project managers in industry requires strategies for the development of the management-related knowledge and skills of the current and future software workforce. Although several approaches help teach the required skills in a university setting, few empirical studies are currently available to characterize and compare their effects. This paper presents results of an externally replicated controlled experiment that evaluates the learning effectiveness of using a process simulation model for educating computer science students in software project management. While the experimental group applies a system dynamics (SD) simulation model, the control group uses the well-known COCOMO model as a predictive tool for project planning. The results of the empirical study indicate that students using the simulation model gain a better understanding about typical behavior patterns of software development projects. The combination of the results from the initial experiment and the replication corroborates this finding. Additional analysis shows that the observed effect can mainly be attributed to the use of the simulation model in combination with a web-based role-play scenario. This finding is strongly supported by information gathered from the debriefing questionnaires of subjects in the experimental group. They consistently rated the simulation-based role-play scenario as a very useful approach for learning about issues in software project management.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdel-Hamid, T. K., and Madnick, S. E. 1991. Software Project Dynamics-an Integrated Approach. Prentice-Hall.
Boehm, B. W. 1981. Software Engineering Economics. Englewood Cliffs: Prentice-Hall.
Boehm, B. W. September 1987. Industrial software metrics top 10 list. IEEE Software: 84-85.
Boehm, B. W., Abts, C., Brown, W. A., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R., Reifer, D. J., and Steece, B. 2000. Software Cost Estimation with COCOMO II. Upper Saddle River: Prentice Hall PTR.
Briand, L. C., Bunse, C., Daly, J. W., and Differding, C. 1997. An experimental comparison of the maintainability of object-oriented and structured design documents. Empirical Software Engineering 2(3): 291-312.
Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Academic Press.
Drappa, A., and Ludewig, J. 1999. Quantitative modeling for the interactive simulation of software projects. Journal of Systems and Software 46: 113-122.
Forrester, J. W. 1971. Principles of Systems. Cambridge: Productivity Press.
Graham, A. K., Morecroft, J. D. W., Senge, P.M., and Sterman, J. D. 1992. Model-supported case studies for management education. European Journal of Operational Research. 59: 151-166.
Kellner, M. I., and Hansen, G. A. January 1989. Software process modeling: A case study. Proc. 22nd Annual Hawaii Int'l Conf. System Sciences.
Lane, D. C. 1995. On a resurgence of management simulation games. Journal of the Operational Research Society. 46: 604-625.
Likert, R. 1932. A technique for the measurement of attitude. Archives of Psychology 22(140).
Lin, C. Y. May 1993. Walking on battlefields: Tools for strategic software management. American programmer: 33-40.
Lin, C. Y., Abdel-Hamid, T., and Sherif, J. S. 1997. Software-engineering process simulation model (SEPS). Journal of Systems and Software 38: 263-277.
Lipsey, M. 1990. Design Sensitivity. Sage Publications.
Madachy, R., and Tarbet, D. June 1999. Case studies in software process modeling with system dynamics. Proc. 2nd Software Process Simulation Modeling Workshop (ProSim'99). Silver Falls, Oregon.
Milling, P. March 1995. Managementsimulation im Prozeß des organizationalen Lernens [organizational Learning and its Support by Management Simulators]. Zeitschrift für Betriebswirtschaft, Ergänzungsheft (supplement) 3/95: Lernende Unternehmen. 93-112. (Also available at URL http://iswww.bwl.unimannheim.de).
Morecroft, J. D. W. 1988. System dynamics and microworlds for policymakers. European Journal of Operational Research 35: 301-320.
Pfahl, D. 2001. An Integrated Approach to Simulation-Based Learning in Support of Strategic and Project Management in Software Organisations. PhD Theses in Experimental Software Engineering, Vol. 8. Stuttgart: Fraunhofer IRB Press.
Pfahl, D., Klemm, M., and Ruhe, G. 2001. A CBT module with integrated simulation component for software project management education and training. Journal of Software and Systems 59: 283-298.
Pfahl, D., Koval, N., and Ruhe, G. April 2001. An experiment for evaluating the effectiveness of using a system dynamics simulation model in software project management education. Proceedings of 7th International Software Metrics Symposium. London, United Kingdom, 97-109.
Richardson, G. P., and Pugh, A. L. 1981. Introduction to System Dynamics Modeling with DYNAMO. Cambridge: Productivity Press.
Sheskin, D. J. 1997. Handbook of Parametric and Nonparametric Statistical Procedures. Boca Raton: CRC Press.
Smith, B. J., Nguyen, N., and Vidale, R. F. May 1993. Death of a software manager: How to avoid career suicide through dynamic software process modeling. American programmer 10-17.
Vennix, J. A. M. 1990. Mental Models and Computer Models-design and evaluation of a computerbased learning environment for policy-making. PhD Thesis, University of Nijmegen.
Wildt, A. R., and Ahtola, O. T. 1978. Analysis of Covariance. Newbury Park: Sage Publications. Sage University Paper Series on Quantitative Applications in the Social Sciences, series no. 07-012.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Pfahl, D., Laitenberger, O., Dorsch, J. et al. An Externally Replicated Experiment for Evaluating the Learning Effectiveness of Using Simulations in Software Project Management Education. Empirical Software Engineering 8, 367–395 (2003). https://doi.org/10.1023/A:1025320418915
Issue Date:
DOI: https://doi.org/10.1023/A:1025320418915