Prototyping a Process Monitoring Experiment

Reviewer: Richard John Botting

Although this paper is hard to read, it has important information for managers, researchers, and programmers. The authors want to find out “what people actually do when they add features to a large software system.” They are planning to automatically collect daily data on a particular software process. Before doing this, they carried out the pilot study described in this paper. Section 1 describes the authors' theory that enhancement can be modeled as a queueing system. Section 2 discusses how to do such experiments. Section 3 shows three advantages of doing a pilot study. First, prototypes improve the tools, and so save money developing them. Second, the results are interesting: programmer time is wasted waiting for data, especially at the beginning and end of an enhancement. An initially orderly “waterfall” becomes chaotic. Redoing work takes a significant amount of time. Third, the prototype data may give a better picture of normal processes than the full experiment. The authors will modify a change management system so that it asks programmers of selected enhancements to record data on their last major task. The programmers will be trained to use a menu-driven tool to do this. The training is “focused on clarifying the process tasks and states.” This means training programmers to think more clearly about the programming process, which should improve it. This desirable outcome makes the data less relevant to other processes.