Quality Rewards

Software Quality has already become an issue of paramount
importance in the IT industry. We will discus the motivation behind the development and
application of the SEI, ISO and MBA models to software organizations. In this context, let
us examine the question 'What is the software problem?'



The software problem can be best summarized by three phrases:



* Hardware revolution



* Current day communication technology, and



* Software capability



The hardware revolution, which is still continuing, is a phenomenon that is responsible
for providing vastly increased computing power to our computers at the rate of doubling
every two to three years while the computers are continuously shrinking in their physical
size and are available at lesser cost. On the other hand, technology development leading
to increased network communication and telecommunication is shrinking the globe into an
interconnected workplace.



Unfortunately, our ability to build software has not kept pace with rapid hardware
development. By shifting from assembly language programming to high-level programming
languages, we have improved our capability probably by a factor of five. By automating
some of the administrative functions of programming, we have increased our ability to
develop larger programs than before by may be another factor of five. Hence, the ability
to develop defect-free software conforming to customer requirements will require much more
than just focusing on the use of software tools like compilers, fourth generation
languages, code generators, the usage of Computer Aided Software Engineering (CASE) tools
and software development paradigms based on the object-oriented design and programming.



The hardware and communication technology developments have impacted the basic thinking
and philosophy behind software development too. We have transited from the early days of
programming when it was assumed that the computer had only one CPU to the days of
multi-programming to parallel software development environment. Programmers are even today
struggling to get rid of the 'Von Newmann mindset' to adjust to current day parallel
computing environments where the way the software process used to develop a solution may
itself be several parallel software processes.

From the above scenario some key conclusions that can be drawn are:



* Complexity is a major issue that goes with any current day software effort



* Experience in building very large software product or solutions reliably is still rare



* Advances in different forms of communications technology is making an impact on the
process to develop software in several fundamental ways. For example:



* It is possible to have development teams spread out across the globe working toward a
common goal of developing software solutions,



* The software runs on a parallel computing environment,



* The software process used to develop is probably having several sub processes that are
run in parallel



* Hardware and software technology today supports online, distributed databases spread
across globally,



* The pressure to provide totally integrated software and systems solutions for a specific
customer requirement is increasing due to competition, leading to a requirement to have
decreased cycle time for providing software solutions,



* The criticality and complexity of the software applications have caused software quality
to become an issue of paramount importance,



* Customers have started demanding perfection in software solutions.



Hence to produce quality software we need to be organized and deploy advanced software
processes within the organization and to our suppliers/vendors. The quality of software
engineering practices will dictate productivity gains and profitability of a software
organization in future. The development and application of all the quality models like
SEI, ISO 9001 and MBA have today provided a direction to the software industry in
grappling with a multitude of complex organizational challenges like management and
technical leadership, handling complexity, process orientation, operational results and
customer satisfaction.



In the 1980's a worldwide quality movement started, under the leadership of the
International Organization of Standards (ISO), which has today gained significant momentum
in all continents. The ISO approach has been to have a very generic and broad quality
standard framework, so as to not limit it to any particular business area or technology
specific area. For the IT/software industry, it finalized on a guideline to help the
process of interpretation of the ISO 90001 standard which is the document ISO 9000-3.



Similarly, the Software Engineering Institute at the Carnegie Mellon University is a
federally funded research and development center sponsored by the Department of Defense of
the US through the Defense Advanced Research Projects Agency. Because software has become
an increasingly critical component of US defense systems, and the demand for quality
software produced on schedule and within budget exceeds its supply, the SEI was
established in December 1984 with a charter to advance the 'state of practice of software
engineering'.



On the same lines, some of the primary objectives for establishing the foundation for the
Malcolm Baldrige National Quality Award by the US Department of Commerce in the late 1980s
is to promote:



* Awareness of quality as an increasingly important element in competitiveness,



* Understanding of the requirements for quality excellence, and



*l Sharing of information on successful quality strategies and the benefits derived from
implementation of these strategies.



Since the basic intention of all the models is quality improvement, let us first define
quality unambiguously.



What is quality?

Quality is simply the conformance to requirements. When people are asked to do it right
the first time, note that the 'it' here refers to quality.



When individuals are asked "What systems are involved in quality?" they
immediately think of the familiar ones such as inspections, tests, reviews and audits.
These systems have a common failing; they are aimed at detecting errors or defects and
fixing them. Such systems are based on the philosophy of appraisal.



The only system that causes quality is prevention. Prevention means eliminating the
potential for error. It involves identification of opportunities for error and taking
actions to eliminate those opportunities.



Prevention is difficult and involves thinking, planning and analyzing processes to
determine where opportunities for error are and then taking actions to ensure that errors
do not happen.



Appraisal on the other hand is easy; it involves hiring a few more inspectors, proof
readers etc. If a few mistakes occur or a customer is dissatisfied we have the mechanism
of causal analysis to blame someone-"If only that had been done this would not
have...".



A prevention system means getting all things straight upfront. 'It' does not mean talking
about a vague concept of 'customer satisfaction'. It means finding out exactly what the
customer needs, establishing formal requirements and then conforming to those requirements
all the time.



Performance standard

The frequent question you would hear in a corridor, "How many times would you expect
me to do things right?". The answer must be "All the time". The standard
therefore that must be communicated is Zero Defect which is symbolic expression for `Do it
right the first time'.



At work, however, a different standard exists. When individuals think that they have an
option of not doing things right the first time, they do not take necessary actions for
improvement. Over the years, conventional approach has 'institutionalized' the
acceptability of errors.



The Zero Defect reflect in our mission statements-"We will deliver defect-free
products to customers, on time."



Achieving operational results through quality

Now we can define the quality standard unambiguously as-"Do it right the first time
and all the time."



Let us re-examine the phrase closely:



Do It means conform to requirements, right means prevent errors always, the first time
means zero defects, no rework needed.



The price of Nonconformance is simply the cost incurred when we fail to do it right the
first time (Crosby).



The SEI CMM framework for software

The SEI's Software Capability Maturity Model characterizes the software process of an
organization by establishing the organization's position in the SEI maturity framework.

The five levels of process maturity in the SEI model have the following general
characteristics.



Initial: Until the process is under statistical control, orderly progress in
process improvement is not possible. While there are many degrees of statistical control
the first step is to achieve rudimentary predictability of schedules and cost.

Repeatable: The organization has achieved a stable process with a repeatable level of
statistical control by initiating rigorous project management of commitments, cost,
schedules and changes.



Defined: The organization has defined the process as a basis for consistent
implementation and better understanding. At this point advanced technology can be usefully
introduced.



Managed: The organization has initiated comprehensive process measurements and
analysis. This is when the most significant quality improvements begin.

Optimizing: The organization now has the foundation for continuing improvement and
optimization of the process.



While the above scheme might look very simple and concise, each level represents a
significantly different way of functioning. There is a complete shift in paradigm (ie, the
way of thinking) within the professionals as the organization evolves through these phases
of improvements.



The complete description of the SEI maturity model is contained in a very detailed SEI CMU
Technical Report (Paulk) entitled "Capability Maturity Model for Software, Version
1.1". This document describes in detail the structure of the Capability Maturity
Model, its operational definition and keys to interpret the CMM and detailed descriptions
of each key process areas (KPAs). This document runs into several hundred pages and is an
excellent application reference document for the Software Engineering Process Group,
Software Project Managers and individuals and teams involved within the organization in
software process definition, software process assessments, software quality assurance and
software process deployment.



A careful reading of the detailed description of the SEI CMM clearly brings out the
tremendous impact the ideas of recognized quality gurus had on the definition of this
model. To mention some of the key names, the SEI CMM has heavily drawn from the work of
Walter Shewart, WE Deming, Joseph Juran and Philip Crosby.



As per the SEI maturity framework, an organization found to be at a particular SEI
maturity level requires to focus its improvement program on the next higher SEI maturity
level's KPAs for improving its software engineering practices. When eventually, the
organization satisfies the requirements for all the KPAs for the next higher SEI level,
then it is said to have evolved its software process to the next level of maturity. For
example, to move from the Initial (1) level to Repeatable (2) level, an organization has
to focus its improvement program on the SEI level 2 KPAs, ie Requirements Management,
Software Project Planning, Software Project Tracking and Oversight, Software Subcontract
Management, Software Quality Assurance and Software Configuration Management.



Each KPA description contains:



* A brief explanation of the KPA,



* the goals for the KPA, and



* the key practices



The key practices are the practices which if performed with the right intent, deliver the
desired results which meet the goals for the KPA.



It is interesting to study at a glance how a set of key practices are structured to fully
meet the goals for a KPA. The key practices can be categorized under five headings.
Towards meeting every goal of a particular KPA satisfactorily, the intents of all the key
practices must be met.



The key practices fall under the following headings:



* Commitment to perform,



* Ability to perform,



* Activities performed,



* Measurement and analysis, and



* Verifying implementation



Commitment to perform and ability to perform is actually the 'Plan' Part, Activities
performed is the 'Do' part, Measurement and Analysis is the 'Check and Act' part of
Deming's famous PDCA cycle. While the verifying implementation is generally a 'check and
act' activity primarily done by the appropriate organizational system, it can be done by
the senior management, project management or software quality assurance professionals with
the primary intention to assure conformance to agreed organizational processes.



Similarly, using Philip Crosby's terminology, the key practices under the heading
Commitment to Perform specifies the 'Performance Standard', those under Ability to Perform
ensures preventive actions and appropriate training and eduction to 'Prevent Defects',
those under Activities Performed are 'Do it right the first time', those under Measurement
and Analysis are 'Measuring the cost of non-conformance' and those under 'Verifying
Implementation' are preventive measures to eliminate cost of non-conformance by root
analysis and process improvements.



Just to provide an idea about the total number of Key Practices defined in the SEI CMM
version 1.1, there are in all about 150 Key Practices under the heading 'Activities
Performed' throughout the detailed definition document.



One of the key characteristics of the SEI CMM model is that it interprets the basis Deming
or Crosby quality model in the context of the software business for various types of
software development, integration, and software systems engineering. The concepts are
equally applicable for small organizations as it is to large projects and large
organizations.



The ISO 9001 framework

First of all, the ISO 9001 QMS is defined very carefully so as to apply very broadly. It
is not a technology or industry specific framework. In fact, to help in the interpretation
of the ISO 9001 framework a documentation by the name ISO 9000-3 has been created which is
named as 'Guidelines for the application of ISO 9001 to the development, supply and
maintenance of software'. It is the guideline document that helps in the interpretation of
the ISO 90001 standard in the context of software development.



The software technology focus is relatively much weaker as per the ISO 9000-3 guideline
document compared to the SEI CMM framework. But the ISO 9000-3 guideline is very strict
about configuration management, document control and document distribution practices. The
scope of the ISO 9001 alongwith ISO 9000-3 is surely broader than the SEI CMM framework
SEI CMM framework addresses supplier management quality systems issues similar to the ISO
9001/ISO 900-3 framework but scope of an SEI assessment will remain limited to the aspects
of businesses that have significant software component. On the contrary, a company with
two interacting departments where one part of the business has significant software
component whereas the other part does not have significant software component can apply
the same ISO 9001 standard across the organization but will be required to apply the ISO
9000-3 guidelines for all the identified software operations.



From the conceptual quality model point of view, SEI CMM model fully implements Philip B
Crosby's model of quality improvement. Right from detailed descriptions of the SEI level 2
KPA till SEI level 5 there is a continuously increasing focus on defect prevention. At the
optimizing level the focus on defect prevention is so much that there is a KPA itself
named as 'Defect Prevention'.



On the contrary the ISO 9001 model does not implement the Philip B Crosby's model of
quality improvement fully. It takes the first few significant steps to ensure that an
organization has a QMS in place, follows a process and there is an in-built system to
continuously improve all the processes. Technical issues like new technology adoption,
effective technology change management are not addressed adequately.



The Malcolm Baldrige Framework

The unique thing about the MBA framework is its very high focus on approach, deployment
and results. There are seven sections as follows:



* Senior executive leadership,



*Information and analysis,



* Strategic quality planning,



* Human resource development and management,



* Management of process quality



* Quality and operational results, and



* Customer focus and satisfaction.



It communicates upfront that there are dynamic relationships amongst all the above areas
of organizational excellence. The `Driver' of this model is the section on `Senior
Executive Leadership' which triggers the improvement process in the areas of information
and analysis, strategic quality planning, human resource development and management and
management of process quality. There is definitely a certain amount of interrelationship
amongst the sections of information and analysis, strategic quality planning, human
resource development and management and management of process quality and all of these
generate feedback to the senior executive leadership processes.



Good work in the areas of senior executive leadership, information and analysis, strategic
quality planning, human resource development and management and management of process
quality results in improved quality and operational results and customer focus and
satisfaction.



The MBA model is actually a way to assess the overall organizational health and
performance. The evaluation consists of clear gradation of the extent of process
definition, deployment and effectiveness.



In some sense the MBA scoring guidelines resembles the philosophy behind the SEI level 1,
2,3,4, and 5 where SEI level 1 corresponds to 0% score, while the SEI level 5 (optimizing)
corresponds to 100% score.



There is no reason why a combination of different frame works cannot work together. It is
required to communicate to the people within the organization what these frame works means
and how we are going to apply the concepts to our organization to make improvements, gain
competitive advantage and aim for complete elimination of the cost of non-conformance.
Different quality models do have a varied degree of emphasis on key challenges facing the
software industry. It is important to have an improvement program in the organization that
addresses the various complex organizational challenges. This might require the
application of more than one framework for quality improvement to be followed
simultaneously.

SEI CMM concept of Key Practices vs Deming and Crosby
models

MBA Scorecard

Excerpted from ISO 9000 One Source

for Software Units, Volume (I).

Courtesy: QAI (India).