Auditing may be as simple as producing an occasional print-out, or reviewing
by query the various changes that have occurred in the system. Or it can
mean compliance with very specific requirements as defined by the US
Department of Defense or the ISO 9003. Either way, knowing that every
change is audited by whom, why and when assures that all requirements can be
met, whether casual or formal.
What is SEI? CMM? ISO? IEEE? ANSI? Will it help?
SEI = 'Software Engineering Institute' at Carnegie-Mellon
University; initiated by the U.S. Defense Department to help improve
software development processes.
CMM = 'Capability Maturity Model', developed by the SEI.
It's a model of 5 levels of organizational 'maturity' that determine
effectiveness in delivering quality software. It is geared to large
organizations such as large U.S. Defense Department contractors. However,
many of the QA processes involved are appropriate to any organization, and
if reasonably applied can be helpful. Organizations can receive CMM ratings
by undergoing assessments by qualified auditors.
Level 1 - characterized by chaos, periodic panics, and heroic
efforts required by individuals to successfully
complete projects. Few if any processes in place;
successes may not be repeatable.
Level 2 - software project tracking, requirements management,
realistic planning, and configuration management
processes are in place; successful practices can
be repeated.
Level 3 - standard software development and maintenance processes
are integrated throughout an organization; a Software
Engineering Process Group is is in place to oversee
software processes, and training programs are used to
ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes,
and products. Project performance is predictable,
and quality is consistently high.
Level 5 - the focus is on continouous process improvement. The
impact of new processes and technologies can be
predicted and effectively implemented when required.
(Perspective on CMM ratings: During 1992-1996 533 organizations
were assessed. Of those, 62% were rated at Level 1, 23% at 2,
13% at 3, 2% at 4, and 0.4% at 5. The median size of
organizations was 100 software engineering/maintenance personnel;
31% of organizations were U.S. federal contractors. For those
rated at Level 1, the most problematical key process area was
in Software Quality Assurance.)
ISO = 'International Organisation for Standards' - The ISO
9001, 9002, and 9003 standards concern quality systems that are assessed by
outside auditors, and they apply to many kinds of production and
manufacturing organizations, not just software. The most comprehensive is
9001, and this is the one most often used by software development
organizations. It covers documentation, design, development, production,
testing, installation, servicing, and other processes. ISO 9000-3 (not the
same as 9003) is a guideline for applying ISO 9001 to software development
organizations. The U.S. version of the ISO 9000 series standards is exactly
the same as the international version, and is called the ANSI/ASQ Q9000
series. The U.S. version can be purchased directly from the ASQ (American
Society for Quality) or the ANSI organizations. To be ISO 9001 certified, a
third-party auditor assesses an organization, and certification is typically
good for about 3 years, after which a complete reassessment is required.
Note that ISO 9000 certification does not necessarily indicate quality
products - it indicates only that documented processes are followed.
(Publication of revised ISO standards are expected in late 2000; see
http://www.iso.ch/9000e/revisionstoc.htm for latest info.)
IEEE = 'Institute of Electrical and Electronics Engineers' -
among other things, creates standards such as 'IEEE Standard for Software
Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software
Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality
Assurance Plans' (IEEE/ANSI Standard 730), and others.
ANSI = 'American National Standards Institute', the primary
industrial standards body in the U.S.; publishes some software-related
standards in conjunction with the IEEE and ASQ (American Society for
Quality).
