The Switchover

Software organizations using ISO

9000 model are often concerned with how ISO 9000 compares with the Capability Maturity

Model (CMM) of the Software Engineering Institute (SEI), Carnegie Mellon University,

Pittsburgh. Concerns typically relate to the difference and similarities between the two

and the equivalence between them. A major issue for organizations using ISO model is the

additional effort that would be required to achieve the various levels of CMM.






The CMM has been evolved by SEI as a model for improving the software process. It is based
on an understanding of the differences between immature and mature software organizations.

The CMM provides software organizations a guidance on how to gain control of their

processes for developing and maintaining software and how to evolve toward a culture of

software engineering and management excellence.

The framework for CMM was inspired

by Crosby's Quality is Free' and adapted by Radice and colleagues working under Watts

Humphrey at IBM. This maturity framework was brought to SEI in 1986. The CMM has been

further developed since then, using inputs from a variety of people from government and

industry.



CMM includes documents which describe the model and detail the key practices of the model.
These documents also contain maturity questionnaires which may be used to assess the

current maturity level of a software organization.






CMM describes the principles and practices for acquiring software process maturity. It is
to be used by software organizations wanting to improve their software processes. The

model is organized into five levels-each level representing a stage in the evolutionary

process, progressing from an ad-hoc and chaotic software process to one which is mature

and disciplined. The framework represents a path of improvements recommended for software

organizations to increase their software process capability.






The five maturity levels in CMM are: initial, repeatable, defined, managed and optimizing.
Each is briefly described below:






INITIAL: Here the software process is adhoc (may even be chaotic), with few
processes being defined and success being dependent on individual capability and effort,

and not being ensured by software processes.






REPEATABLE: Here basic management practices are established. Cost, functionality
and schedules are tracked. At this level, success is 'repeatable' for similar projects and

applications as there is process discipline.






DEFINE: In this level, software processes-duly defined and documented-are
integrated for the organization, both for management and engineering aspects. In an

organization at this level, all the projects use an organization's standard processes,

duly tailored for the individual project.






MANAGED: Here quantitative measurement and control are used. The quality of
software processes and products is measured and this measurement is used to understand and

control software processes and products.






OPTIMIZING: Focus here is on continuous process improvement, quantitative feedbacks
and using innovative ideas and technologies.






As can be seen, level 1 is the lowest, hence called 'initial'. An organization at the
initial level trying to evolve its software processes works on various aspects to reach

level 2, then level 3 and so on. CMM uses the concept of key process areas (KPAs) to

describe this evolution. Each KPA identifies a cluster of related activities that, when

performed collectively, achieve a set of goals that enhance process capability. Except for

level 1, each maturity level is decomposed into a set of KPAs which have to be addressed

to reach that maturity level.






KPAs for level 2 focus on basic software project concerns. Without these KPAs, projects
cannot be executed with 'repeatable' success. Minimum software project management requires

addressing these areas-starting from requirements management and moving on to project

planning, tracking, quality assurance and configuration control. Subcontracts are also

need to be managed.






Level 3 KPAs address both project and organizational issues. At this level, the
organization has to establish the required infrastructure and institutionalize effective

practices. They include an organizational perspective to processes and training.

Improvement in management aspects is through KPAs which integrate software management and

coordinate between groups. Software engineering for products and inclusion of peer reviews

are also needed to bring the organizations processes to the 'defined' level.






At level 4, a quantitative understanding is used for improving management and
organizational processes and quality management is used to improve the engineering

process. At the highest level, level 5, continuous and measurable process improvement is

the focus through technological and process change management and defect prevention.






The description of each KPA in CMM is done with:


* A brief description of the KPA.


* Goals of the KPA: These define the scope, boundaries and intent of the area and can be
used to determine whether the organization has effectively implemented the KPA.



* Key practices for the KPA: A key practice describes what is to be done.


* Common features are used to organize the key practices into sections. These features are
attributes that indicate whether the implementation and institutionalization of a KPA are

effective, repeatable and lasting. There are five common features used as sections in each

KPA:



- Commitment to perform


- Ability to perform


- Activities performed


- Measurement and analysis


- Verifying implementation


- Practices in 'activities performed' describe what must be implemented to establish a
process capability. Other practices form the basis by which the organization can

institutionalize the practices described in the activities performed.



* Sub-practices or subordinate key practices are listed below key practices and describe
what is expected to be implemented for the key practice. They help to determine whether a

key practice is implemented or not.



* Supplementary information is also provided along with subpractices to provide examples,
elaborations and references for the key practices.






Software process assessments and software capability evaluations are two methods used to
appraise the maturity of an organization's execution of the software process. Maturity

questionnaires are used by suitably qualified/trained assessor team.





ISO 9000 and CMM approaches



ISO 9000 is a generic series of standards and is applicable to all industries, and not

restricted to organizations developing and maintaining software. For software
organizations, certification in ISO is against ISO 9001, a generic standard. CMM, on the

other hand, is specific to software organizations. While ISO checks for 'quality' against

overall customer requirements, CMM considers only the software aspect and is therefore

more limited in its guidance and its requirements. If the customer requirements include a

non-software component, CMM does not cover this aspect of the requirement. On the other

hand, being generic, ISO 9001 does not specially treat the aspects peculiar to software.






ISO 9000 does recognize that some special interpretations are needed for software. The ISO
9000 series includes guidelines in the form of ISO 9000-3, which has been written

specifically for software organizations. Further, guidelines are often taken from the

TickIT documents which are written for auditors auditing software companies-these

guidelines are fairly well accepted and provide additional details relevant for software.






The degree of detail of the two models is quite different. ISO 9001 (and even the
accompanying guideline, ISO 9000-3) are small and concise whereas the documents in CMM are

fairly large-the document describing the key practices in CMM is about 350 pages. The

brief nature of the ISO 9000 standards means that there is more left to interpretation.

Whereas the basic requirement of the ISO 9001 standard is clear. Auditors differ in their

interpretation of and insistence on 'implicit' requirements, or the 'spirit' of the

standards. Organizations which have obtained certification may therefore be far apart in

software maturity as auditors may have differed in their interpretation of the

requirement. CMM, on the other hand, is detailed, and prevents (to a large extent)

differences in interpretation.



Both ISO 9000 and CMM are process-based models-they focus on standardizing and improving
the process. Neither of them prescribes product standards. Both focus on 'what' is

required rather than 'how' to implement a requirement. The way a particular quality

requirement is to be met is left to the organization-the standards are non-prescriptive.

However, being more detailed and software specific, CMM does specify at places the tool to

be used (for example, it requires 'peer reviews' whereas ISO allows use of any 'suitable'

method for review) and is therefore more 'prescriptive.'






Both models are concerned with organizational and project-level concerns. In general, both
models cover similar areas and have similar concerns. However, ISO does have a somewhat

greater emphasis on external contractual situations where CMM is more internal in its

concerns.






The two models are structured differently and therefore are difficult to compare-it is not
possible to do a point-to-point comparison. The major point of debate while comparing the

general approach of the two models is the 'continual improvement' aspect. CMM is divided

into levels, and an organization uses these levels as steps in its process of improving

quality. The focus is therefore on improvement-from level to level. Continuous review and

improvement are built into the model. There is, however, some debate on whether ISO also

seeks continuous improvement. Whereas the 'spirit' of the ISO 9000 standard does include

continuous improvement, this is not clearly stated or directly enforceable and is hence

subject to the interpretation of the auditors. Even though organizations may be sincere

about continuous improvement and auditors may also be alert about it, the model does not

have adequate emphasis or enough stated requirements on this aspect. An organization

serious about continuous improvement will need more guidelines than provided in the ISO

9000 series; an organization just concerned with maintaining its standard and not with

improving would also be able to maintain its certification in many cases. ISO 9000, in

general, is seen as requiring a 'minimum acceptable' level for certification. It is a

single-level standard (you are either certified or not) with surveillance to check that

the standard is maintained.






CMM maturity levels and ISO requirements


An organization familiar with ISO 9000 and aiming at or having received ISO certification
would be typically interested in knowing which areas it needs to address for achieving the

various CMM maturity levels. ISO 9001 and CMM are structured differently, there is no

direct mapping possible between the two models.






Level: Repeatable


REQUIREMENTS MANAGEMENT: For this KPA, systems requirements allocated to software
are controlled. System requirements form the baseline for software engineering and

management use and all plans, products and activities are to be kept consistent with them.

ISO does not explicitly specify that these requirements form the basis for the software

development plan and for developing the software.






SOFTWARE PROJECT PLANNING: The goals in this KPA include documenting estimates for
use in project planning and tracking. Project activities and commitments have to be

planned and documented and affected parties have to agree to the commitments related to

the project.






In addition to the ISO requirements, CMM requires that software project planning be
initiated in the early stages of and in parallel with the overall project planning. CMM is

emphatic on estimates being derived from documented procedures for the product sizes,

project efforts and costs.






It also requires that information recorded on software planning include all data needed to
reconstruct the estimate and to assess whether the estimate is reasonable, and that such

data be managed and controlled. ISO has no corresponding requirement.






SOFTWARE PROJECT TRACKING AND OVERSIGHT: The tracking of actual results and
performance and taking corrective actions for deviations from plan form part of this

area's goals. Changes to software commitments are to be agreed to by affected parties.

This area is covered very well in ISO.






SOFTWARE SUBCONTRACT MANAGEMENT: CMM requires that the prime contractor select
qualified software subcontractors, that they agree to their commitments to each other and

maintain continuous communication. The prime contractor needs to track actual results and

performance of the subcontractor against the agreed commitments.






ISO does not cover approval of the subcontractor's development plan by the prime
contractor, or the use of this for tracking and communicating status. Periodic

status/coordination reviews of the prime contractor's management with the subcontractor's

management are not required in ISO. CMM also requires monitoring of the software

configuration management at the subcontractor level.



SOFTWARE QUALITY ASSURANCE: CMM's goals for this KPA are that software quality assurance
activities are planned; adhere to applicable standards and procedures; requirements are

objectively verified; affected parties are informed of the quality assurance activities

and results; and unresolvable non-compliances addressed to senior management.






While ISO covers most of the activities to be performed in this area, it does not
prescribe that the SQA group should participate in the preparation and review of the

project development plan, standards and procedures.






SOFTWARE CONFIGURATION MANAGEMENT: Here, CMM goals are that the software
configuration management activities are planned, selected software work products are

identified, controlled, and available, changes are controlled, and affected parties are

informed of the status and contents of the baseline.



Activities required for establishing a software configuration management library system as
a repository of software baselines is an aspect which an ISO compliant organization would

need to look into.





Level: Defined


ORGANIZATION PROCESS FOCUS: For this KPA, CMM goals are that software process

development and improvement activities be coordinated across the organization, strategies
and weakness of processes identified, related to a process standard, and that
organization-linked process development and improvement activities be planned.






While some activities of this KPA are covered in ISO, however, ISO-compliant organization
would need to work on organization-level coordination and the use of the organization

software process database. Monitoring and evaluating new processes, methods and tools used

in parts of the organization and transferring them to other parts of the organization is

not covered in ISO. Organization-level coordination of training for an organization and

project software processes is also not there in ISO. Another activity to be performed in

CMM is informing groups that are implementing software processes about

organization/project activities in the area of software process development/improvement.

This is not present in ISO.






ORGANIZATION PROCESS DEFINITION: CMM goals for this area include developing a
standard software process and maintaining it for the organization and that information

related to its use by software projects collected, reviewed and made available.






ISO does not require documenting and maintaining description of software life-cycles that
are approved for use nor does it require having guidelines or criteria for the projects to

tailor these.






TRAINING PROGRAM: Training activities are to be planned and training to be provided
to develop skills/knowledge required for performing software management/technical roles.

Individuals need to receive training necessary to perform their roles.






Unlike CMM, ISO has no requirement for a standard for training courses prepared at
organization level nor does it require that training for the organization be performed

according to an organization's training plan.






INTEGRATED SOFTWARE MANAGEMENT: The organization standard software process is
tailored to get the project's software process and the project is planned and managed

according to this.






Most activities in this are not part of ISO-such as use of software processes database for
planning/estimation, managing product size, efforts and costs, resources, dependencies

etc. according to documented procedures. This area needs to be looked into carefully by

ISO-compliant organizations aiming at level 3 of CMM.






SOFTWARE PRODUCT ENGINEERING: The goal of this area is that software engineering
tasks are defined, integrated and consistently performed to produce software and that

products are kept consistent with each other.






While most activities in this are covered in ISO, CMM also mentions some specific aspects
of 'coding' (such as criteria for sequence of development and peer reviews) that are not

in ISO.






INTERGROUP COORDINATION: Agreement on the customer requirements by affected groups,
agreement on commitments and identification, and tracking and resolving inter-group issues

are the goals of this KPA. This is reasonably covered in ISO.






REVIEWS: Planning peer review activities and identifying removing defects in
software work products are the broad goals of this KPA. ISO 9001 is not specific on 'peer

reviews' though guidance documents suggest these. ISO does not cover planning 'peer

reviews' and documentation of the plan.






Level: Managed


QUANTITATIVE PROCESS MANAGEMENT: In this KPA, the goals are that quantitative
process management activities are planned, project defined software processes are

controlled quantitatively and process capability of the organization standard software

process is known in quantitative terms.






The earlier ISO 9001 version had little in the area of statistical/quantitative
techniques. While the 1994 release does include the need for statistical techniques, this

KPA will need attention for organizations aiming at level 4.






SOFTWARE QUALITY MANAGEMENT: Planning project's software quality management
activities, defining measurable goals for software product quality and quantifying and

managing actual progress are the goals here.






Whereas the 1994 ISO-9001 release has increased its emphasis on the use of statistical
techniques. CMM activities such as the measurement and analysis of software product

quality on an 'event-driven' basis and allocation of quantitative quality goals to

subcontractors will require attention.






Level: Optimizing


DEFECT PREVENTION: For this area, defect prevention activities should be planned
and common causes of defects sought, identified, prioritized and systematically

eliminated.






While ISO places a high emphasis on defects, CMM has an activity related to the 'kick off
meeting' which will need attention. Periodic circulation of feedback on status and results

of defect prevention activities is another which is a fairly elaborate activity in CMM.






TECHNOLOGY CHANGE MANAGEMENT: Planning for incorporating new technology, evaluating
effects on quality and productivity, and transferring new technology to normal practice

are the broad goals of this KPA. This aspect is covered in a limited way by ISO and will

need to be looked into in detail by an ISO-compliant organization aiming at CMM level 5.






PROCESS CHANGE MANAGEMENT: Planning continuous process improvements, having
organization-wide participation in this and continually improving organization standard

software process and project defined software process is the focus. This aspect is covered

only in a very limited way by ISO and will need to be looked into in detail by an

ISO-compliant organization aiming at CMM level 5.






A rundown


ISO and CMM have both commonalties and differences, but direct comparison between the two
is not possible as the two are structured differently and vary greatly in the degree of

detail. Also, there is a significant degree of 'interpretation' involved, especially in

the ISO standard.






While it is possible that organizations that are still at level 1 of CMM have got ISO
certification, this is not typically expected if the auditors are using the

software-specific guidance for software (ISO 9000-3 and TickIT auditor's guidance) and

going by the spirit of ISO. Organizations that have obtained ISO certification and have

been successfully undergoing surveillance are likely to be performing most of the

activities of the KPAs of level 2 and quite a number of level 3. They would also be

performing some activities of higher levels. Such organizations would require little

effort to achieve level 2. Achieving level 3 would require a critical examination of the

gaps and some additional effort. Other levels will require more attention and effort.






Over the time, as these quality models evolve and incorporate feedback and change requests
from users, the differences are likely to narrow and the possibilities of ambiguities in

interpretation will reduce.






Excerpted from ISO 9000 One Source for Software Units, Volume II.


Courtesy: QAI (India).