Harmonization of ISO/IEC 9001: 2000 and CMMI-DEV: from a theoretical comparison to a real case application

Reviewer: Frank Pospiech

Given the rather large number of software quality frameworks and standards, it is becoming increasingly difficult for practitioners to select the "right" one. Even worse, different customers or governmental agencies might require compliance to different standards. The authors of this paper address a very interesting topic of great practical value. In this paper, a crucial question arises: How does a developer comply with different standards without having to engage in unnecessary and significant effort__?__ To illustrate their approach, the authors choose two very popular standards and frameworks: ISO 9001:2008 and Capability Maturity Model Integration for Development (CMMI-DEV). Their approach sounds quite simple and straightforward: Compare the two models by verifying whether or not, and to what degree, the elements of model 1 are already part of model 2. In other words, they determine how much of model 2 is already in place if an organization is already complying with model 1. However, as usual, the problems arise in the details of how actually to do this. And this is the contribution of this paper: the harmonization process it proposes is a good theoretical approach. Luckily, it is not only theoretical, but proven in a practical case study with small- and medium-sized enterprises (SMEs) (step 2 of the harmonization process). Using their approach, the authors compare the ISO 9001:2008 and CMMI-DEV models. It would be quite interesting to have similar comparison results for other standards or frameworks, such as SPICE (ISO 26262). It would also be great if the comparison results could be published in a paper like this, so they could be checked and approved by the Software Engineering Institute (SEI) and the International Organization for Standardization (ISO), the respective owners of CMMI-DEV and ISO 9001:2008. From a practical point of view, of course, step 2, the application of the mapping to a practical case, is of great interest. It's an excellent idea to use Basili's goal-question-metrics (GQM) method to move from a (source) quality management system (QMS) complying with framework A to a target QMS complying still with A and additionally with framework B. What the authors say about how this is accomplished sounds very reasonable. However, it would have been interesting to get some idea of the amount of effort that is actually required for performing step 2. By the way, the application subprocess (step 2) in the harmonization process, as proposed by the authors, might be used in another way. Quite often, companies lack good question lists to practically assess their compliance with a dedicated standard or framework, which is close to the company's own business goals. Here, the GQM-based application subprocess might provide excellent guidance. Maybe there is some room for further investigation, besides the directions as given in the "Conclusions and Future Work" section. In summary, this paper should interest anyone who works on the practical implementation of service, platform, and infrastructure (SPI) models in a real organization. Online Computing Reviews Service