tl9000M3 0E
tl9000M3 0E
tl9000M3 0E
Telecommunications Forum
(QuEST Forum)
TL 9000
Quality Management System
Measurements Handbook
Release 3.0
Copyright
To the memory of
Terry Blok
Unisys
Hank Malec
3Com
Foreword
Foreword
The TL 9000 Quality Management System Measurements Handbook was
prepared in a cooperative effort by the members of the Quality Excellence for
Suppliers of Telecommunications (QuEST) Forum. From the outset the QuEST
Forums goal has been to develop and maintain a consistent set of quality
system requirements and measurements that, when implemented, will help
provide telecommunications customers with faster, better and more cost-effective
services.
This book complements the TL 9000 Quality Management System Requirements
Handbook with measurements that reflect the performance of the industry and its
products. QuEST Forum members, including service providers and suppliers,
utilize measurements that are collected under the provisions of this handbook to
improve their processes. By improving processes, the industry becomes more
efficient and telecommunications customers, worldwide, derive the benefit of
improved services.
The QuEST Forum is pleased to present this book in a common spirit of
delighting our customers.
Signature on File
Isabelle Courville, Bell Canada
Preface
Preface
The Quality Excellence for Suppliers of Telecommunications Forum (QuEST
Forum) was founded to foster continued improvements to the quality and
reliability of telecommunications service. The founders took the critical initial step
of establishing a common set of quality management system requirements and
measurements by creating the TL 9000 Quality Management System
Requirements Handbook and the TL 9000 Quality Management System
Measurements Handbook. These handbooks are the result of a cooperative
effort among members of the telecommunications industry.
The work of the QuEST Forum yields benefits to customers, their subscribers,
and their suppliers. Membership is composed of telecommunication Service
Providers, Suppliers, and Liaisons. Members fund and participate in the QuEST
Forum, have defined voting rights, and are expected to contribute to the work of
the QuEST Forum. Members vote on adoption of the TL 9000 structure, content,
administration, and other questions coming before the QuEST Forum.
The QuEST Forum establishes and maintains a common set of quality
management system requirements and measurements built on currently used
industry standards including ISO 9001:2000. The requirements and
measurements promote consistency and efficiency, reduce redundancy and
improve customer satisfaction. They also enable suppliers to improve quality
and reliability, reduce costs, and increase competitiveness.
Acknowledgements
Acknowledgements
The strength of the QuEST Forum is the outstanding capabilities and
commitment of the members who represent their respective organizations at the
QuEST Forum and Work Group Meetings. This exceptional talent produced the
first TL 9000 Quality Management System Measurements Handbook in record
time and now has completed this major update in less than one year. Individuals
whose companies were customers, suppliers, and competitors accomplished the
update of the Requirements handbook through extraordinary teamwork.
This outstanding accomplishment was facilitated in partnership with the American
Society of Quality (ASQ) and The University of Texas at Dallas (UTD). Special
thanks for their constant support and encouragement during the last three years
of our development to Dr. Bill Osborne, Dean and Dr. Douglas E. Harris,
Associate Dean, Erik Jonsson School of Engineering and Computer Science and
to Paul Borawski, ASQ Executive Director and Brian LeHouillier, ASQ Director,
Programs & Operations.
Personally, and for the entire QuEST Forum, I would like to thank the following
QuEST Forum individuals and companies of the Measurements and Oversight
work groups for their direct contributions to this update of the TL 9000 Quality
Management System Measurements Handbook.
Jack Pompeo
QuEST Forum Project Director
Contributors:
ADTRAN
Charles ODonnell
Advanced Fibre
Communications
Mark Fischer
Mark Hodges
Rhonda Sator
Chair
Vice-Chair
Secretary
Time Keeper
SME
Rick Werth
Matt Lindsay
John Walz
Jeffery Rose
Richard Morrow
Alcatel
Jim Ko
Peter Loew
June Miller
Steven Quigley
Tab Rabalao
Tom Yohe
SBC
Tellabs Operations
Lucent Technologies
Sumitomo Electric Lightwave
The University of Texas at Dallas
Antec
Bob Lichkay
Rob Lindner
Astec Advanced Power
Systems
Roger Daunais
Andre Lapointe
Acknowledgements
AT&T
Kathy Parker
Michael Server
Bell Canada
Jean-Normand Drouin
BellSouth
Telecommunications
Ed Ballington
Mort Burnett
Tex Prater
Boston Communications
Group
Tareq Rahman
British Telecommunications
Steve Dickens
Celestica Corporation
Paul Pettinger
Charles Industries
Angelo DiMonte
Victor Potent
CommWorks
James Oates
Comverse Network Systems
Bruce Rozett
Elizabeth Tracy
Corning
Kevin Calhoun
Steve Cooper
CTDI
Jim McCormick
Ericsson
Victor Sandoval
Excel Partnership
Steve Obolewicz
Dave Sanicola
Flextronics
Johnny Hancock
Fujitsu
Tom Garrison
Hummingbird
Mary Harmer
Qwest
Jerry Keintz
Don Wilford
SBC
Jim Lankford
Vuong Phi
Scott Suko
Rick Werth
Siemens ICN
Ken Koffman
Mark Young
Sumitomo Electric Lightwave
Jeffrey Rose
Symmetricom
Jack Riskey
Tekelec
Angela Hall
Michael Willis
Telamon
William McLinn
Telcordia Technologies
Debbie Hearn
Tellabs Operations
Duke Dahmen
Denise Echols
Matt Lindsay
Telmar Network Technology
Jerry Constance
The University of Texas at
Dallas
Richard Morrow
Unisys
Richard Geesaman
Verizon Communications
Galen Aycock
Alan Beaudry
Brendan Pelan
Ron Resh
WorldCom
Robert Paschke
Xerox Corporation
Brian Fannon
Chair
Vice-Chair
Secretary
Contributors:
ADC Telecommunications
Randy Pezon
Joe Taylor
Ron Basque
Kim Hauswirth
ADTRAN
Randal Whorton
Tellabs Operations
Complas
American Society for Quality
Alcatel
Dave Aiken
Ron Luttrull
Acknowledgements
Independent Association of
Accredited Registrars
Chris Shillito
Marconi
Donna Reinsch
Motorola
Greg Feldman
Nortel Networks
Jeff Harpe
Perry Johnson Registrars
Nicole Martin
SBC
Jim McDonnell
Siemens ICN
Ken Koffman
Stat-A-Matrix
Rich Watts
Tekelec
Ben Crane
Tellabs Operations
Joe Taylor
The QuEST Forum benefits from the continued and dedicated service of many
individuals working towards the goals of the QuEST Forum. Without these
individuals and their companys support, the QuEST Forum would not be
successful in ensuring that the quality of telecommunication services to the enduser keep pace with changing technological opportunities in the twenty-first
century.
A Board of Directors guides the QuEST Forum activities through a strategic plan,
which is implemented by the work groups. The Measurements and Oversight
work groups are credited for producing this document and they would like to
recognize the individuals and companies that participated in the other work
groups for providing invaluable service in support of the overall QuEST Forum
Mission.
Chair
Vice-Chair
Vice-Chair
Secretary/SME
Contributors:
Agilent Technologies
John Murray
Alcatel
Don Brown
Ian Mackie
AT&T
Robert Gray
BellSouth
Telecommunications
Irv Briks
British Telecommunications
Alex Cuthbertson
Mark Webster
Gene Hutchison
Mary Hattrick
Don Brown
Tom Withey
SBC
Marconi
Alcatel
The University of Texas at Dallas
Infonet
Dan Russ
Lucent Technologies
Sandford Liebesman
Marconi
Mary M. Hattrick
Motorola
Greg Feldman
Nortel Networks
Daniel Proffit
Christopher West
Acknowledgements
SBC
Gene Hutchison
Stephen Stroup
Telkom South Africa
Mike Donald
Tellabs Operations
Mari Silvenius
Telmar Network Technology
Gary McMullin
Co-Chair
Co-Chair
Contributors:
Alcatel
Ron Luttrull
Corning
Len Young
Jim McDonnell
Len Young
SBC
Corning
ECI Telecom
Misha Ptak
Motorola
Greg Feldman
Nortel Networks
Jeff Harpe
SBC
Jim McDonnell
Tellabs Operations
Joe Taylor
Chair
Vice-Chair
Contributors:
ADC Telecommunications
Jerry Lefever
Randy Pezon
ADTRAN
Randal Whorton
Alcatel
Dave Aitken
Ron Luttrull
Jack Pompeo
Ashok Dandekar
TeleCentric
Fujitsu
Complas
Ronald Basque
Corning
Joel Reece
Fujitsu
Ashok Dandekar
Marconi
Donna Reinsch
Contributors:
3M Telecom Systems
Thierno Diallo
ADTRAN
Charles O'Donnell
Chair
Vice-Chair(2000)
Vice-Chair(1999)
Secretary
Time Keeper
SME(2000)
SME(1999)
Brendan Pelan
Debbie Hearn
Matt Lindsay
Tama McBride
Jeffery Rose
Richard Morrow
Bob Brigham
Advanced Fibre
Communications
Mark Hodges
Rhonda Sator
Verizon
Telcordia
Tellabs Operations, Inc.
Motorola
Sumitomo Electric Lightwave
The University of Texas at Dallas
Telcordia
Alcatel
Chandan Banerjee
Ian Mackie
Mark Moore
Mike Rippe
Tom Yohe
Acknowledgements
Antec
Rob Lindner
Bob Lichkay
Bill Taylor
Astec Advanced Power
Systems
Andre Lapointe
AT&T
Kathy Parker
Michael Server
Bell Canada
Jean-Normand Drouin
Jean-Pierre Quoibion
BellSouth
Telecommunications
Tex Prater
Joel Sullivan
British Telecommunications
Steve Dickens
Charles Industries
Angelo DiMonte
Victor Potent
Chatsworth Products
Edward Gaicki
CommWorks
Laura Coplon
Jim Oates
David Stahl
Comverse Network Systems
Bruce Rozett
Elizabeth Tracy
Corning
Steve Cooper
Len Young
Entela
Ralph Stowe
Ericsson
Victor Sandoval
Excel Partnership
Steve Obolewicz
Dave Sanicola
Flextronics
Johnny Hancock
Fujitsu
Doug McCullough
Hekimian Laboratories
Robin Williams
Liebert Corporation
Larry Ables
Tom Baldrick
Dale Carpenter
David Giannamore
Lucent Technologies
Ruth A. Harman
Sandford Liebesman
John Wronka
John Walz
Marconi
Bret Barclay
Don Denk
Mary Hattrick
Beverly McClain
Harold Morrison
Jennifer Vidos
Motorola
Mary Demmert
Tama McBride
Jim Osman
NEC America
David Barski
Nokia
Karen Rawson
Doug Hall
Nortel Networks
Robert Oakley
Richard Pierrie
Paceon
Jim Shields
QUALCOMM
Grace Weaver
Qwest
Don Wilford
Jerry Keintz
SBC
Ed Franck
Jim Lankford
Judy Przekop
Steve Stroup
Rick Werth
Siemens ICN
Pat Muirragui
Tom West
Sorrento Networks
Richard Lycette
Sprint
Tim Dinneen
Sumitomo Electric Lightwave
Gary Bishop
Jeffrey Rose
Symmetricom
Donna Schilling
Jack Riskey
Telcordia Technologies
Bob Brigham
Debbie Hearn
John Russell
Leslie Wolfe
TeleCentric
Jack Pompeo
Tellabs Operations
Matt Lindsay
Telmar Network
Technology
Jerry Constance
Gary McMullin
The University of Texas at
Dallas
Richard Morrow
Tyco Electronics
Greg Blount
Verizon Communications
Alan Beaudry
Brendan Pelan
Worldcom
Robert Paschke
John Rosenow
Chair
Vice-Chair
David Briggs
Greg Lilly
Agilent Technologies
Brent Wahl
Alcatel
Phil Dudley
KARLEE
Nortel Networks
Artesyn Technologies
Scott Ireland
Michael Sullivan
AT&T
Mike Server
Acknowledgements
Graybar Electric
Jack Evans
IECQ/ECCB
Charles Packard
JDS Uniphase
David Hall
KARLEE
David Briggs
Liebert
Larry Ables
Hang Tan
Lucent Technologies
Andrea Long
Mike Musky
Manly-Williams
John Manly
Marconi
John Wheeler
Masterwork Electronics
Scott Woods
Motorola
Jim Osman
Network & Cable Products
Jay Chenault
Corky Roberts
Nokia
Douglas Hall
Nortel Networks
Greg Lilly
NQA
Chris Mooney
Pulsecom
Robert Hungate
QUALCOMM
Grace Weaver
SCI Systems
Ken Crane
Stat-A-Matrix
Hal English
Superior Telecommunications
Jules Fijux
Eric Perry
Telcordia Technologies
John Russell
TeleCentric
Jim Carmichael
Tyco Electronics
Greg Masciana
Verizon
Galen Aycock
Chair
Vice-Chair
Rosemarie Moskow
Jeff Harpe
Nortel Networks
Jeff Harpe
Pulsecom
Misha Ptak
SBC
Jim McCormick
Rosemarie Moskow
SBC
Nortel Networks
Stat-A-Matrix
Paul Berman
Jim Gerard
telcobuy.com
Shannon Kohlenberger
Monica Eskridge
Table of Contents
Table of Contents
SECTION 1
INTRODUCTION
1-1
SECTION 2
STRUCTURE
2-1
SECTION 3
MEASUREMENTS PROCESSING,
USAGE AND RESPONSIBILITIES
3-1
SECTION 4
4-1
SECTION 5
COMMON MEASUREMENTS
5-1
5.1
5.2
5.3
5.4
SECTION 6
5-1
5-9
5-17
5-24
6-1
6-1
HARDWARE MEASUREMENTS
7-1
7-1
SOFTWARE MEASUREMENTS
8-1
8-1
SERVICES MEASUREMENTS
9-1
9-1
APPENDIX A
A-1
APPENDIX B
B-1
SECTION 7
SECTION 8
SECTION 9
GLOSSARY
BIBLIOGRAPHY
1
1
Table of Contents
List of Figures
FIGURE 2.1-1 THE TL 9000 MODEL
2-1
2-2
7-7
List of Tables
List of Tables
Table 5.1-1
Table 5.1-2
Table 5.1-3
Table 5.1-4
Table 5.1-5
Table 5.1-6
Table 5.1-7
Table 5.1-8
Table 5.2-1
Table 5.2-2
Table 5.2-3
Table 5.2-4
Table 5.2-5
Table 5.2-6
Table 5.2-7
Table 5.2-8
Table 5.2-9
Table 5.3-1
Table 5.3-2
Table 5.3-3
Table 5.3-4
Table 5.3-5
Table 5.3-6
Table 5.3-7
Table 5.3-8
Table 5.4-1
Table 5.4-2
Table 5.4-3
Table 5.4-4
Table 5.4-5
Table 5.4-6
5-4
5-4
5-4
5-5
5-6
5-7
5-7
5-8
5-11
5-12
5-12
5-13
5-15
5-15
5-16
5-16
5-16
5-19
5-19
5-20
5-20
5-22
5-22
5-23
5-23
5-26
5-27
5-28
5-29
5-30
5-31
List of Tables
Table 6.1-1
Table 6.1-2
Table 6.1-3
Table 6.1-4
Table 6.1-5
Table 6.1-6
Table 6.1-7
Table 6.1-8
Table 6.1-9
Table 6.1-10
Table 7.1-1
Table 7.1-2
Table 7.1-3
Table 7.1-4
Table 8.1.5-1
Table 8.1.5-2
Table 8.1.5-3
Table 8.1.5-4
Table 8.1.5-5
Table 8.1.5-6
Table 8.1.6-1
Table 8.1.6-2
Table 8.1.6-3
Table 8.1.6-4
Table 8.1.6-5
Table 8.1.6-6
Table 8.1.7-1
Table 8.1.7-2
Table 8.1.7-3
Table 8.1.7-4
6-4
6-5
6-5
6-8
6-9
6-9
6-12
6-12
6-13
6-13
7-5
7-5
7-7
7-12
8-5
8-6
8-6
8-7
8-8
8-9
8-12
8-13
8-13
8-14
8-15
8-16
8-19
8-19
8-20
8-20
List of Tables
Table 8.1.7-5
Table 8.1.7-6
Table 9.1-1
Table 9.1-2
Table 9.1-3
Table 9.1-4
Table 9.1-5
Table 9.1-6
Table 9.1-7
Table 9.1-8
Table 9.1-9
Table 9.1-10
Table A-1
Table A-2
Table A-3
Table A-4
Table A-5
8-21
8-22
9-2
9-3
9-4
9-4
9-5
9-5
9-6
9-6
9-6
9-7
A-3
A-23
A-35
A-36
A-37
Section 1 - Introduction
Section 1
Introduction
The TL 9000 handbooks (the TL 9000 Quality Management System
Requirements Handbook and the TL 9000 Quality Management System
Measurements Handbook) are designed specifically for the telecommunications
industry to document industry quality management system requirements and
measurements.
The TL 9000 Quality Management System Requirements Handbook establishes
a common set of quality management system requirements for suppliers of
telecommunications products: hardware, software, and services. The
requirements built on existing industry standards, including ISO 9001. The
TL 9000 Quality Management System Measurements Handbook defines a
minimum set of performance measurements. The measurements are selected to
guide progress and evaluate results of quality management system
implementation.
1.1 Goals
1.2 Purpose
1.3 Benefits of
Implementation
Foster quality management systems that effectively and efficiently protect the
integrity and use of telecommunications products: hardware, software, and
services,
Section 1 - Introduction
Uniform measurements,
The QuEST Forum maintains compatibility with other sets of requirements and
1.5 Developing and and maintenance of the TL 9000 handbooks. Change requests for the
Maintaining the
handbooks, following initial publication, are to be submitted to the QuEST Forum
Handbook(s)
Administrator. Any user of the handbooks may submit change requests.
Change requests will be forwarded to the appropriate handbook section
chairperson by the QuEST Forum Administrator and will be considered for the
next revision. A change request/feedback form is available at the QuEST Forum
web site (http://www.questforum.org/).
Final approval of all changes to TL 9000 handbooks will be by vote of the QuEST
Forum voting members in accordance with the QuEST Forums bylaws. Reissue of the TL 9000 handbooks will be determined by the QuEST Forum, but not
to exceed five years following the last issue date. When the QuEST Forum
determines there are changes necessary in TL 9000 that could impact third party
registration, then addenda or similar communication mechanisms will be
employed to inform the industry of corrections and updates to the TL 9000
handbooks.
Section 2 Structure
Section 2
2.1 Overall
Structure
Structure
TL 9000 is structured in layers (see Figure 2.1-1):
T
L
9
0
0
0
2.2 Terminology
In this handbook the term supplier refers to the organization pursuing TL 9000
implementation, conformance, and/or registration.
Section 2 Structure
Figure 2.3-1 illustrates the data flow and usage of TL 9000 Quality Management
System Measurements as described in this handbook.
Customer
Supplier
M RS Reported
TL 9000 Data
Continuous
Improvement
Programs
QuEST Forum
Measurements
Repository
System (MRS)
Industry Statistics
Web Site
Section 3
3.1 Requirements
for Measurements
Usage
3.2 Principles of
Measurements
Processing
b. Valid reasons for the exclusion of specific measurements from the scope of
registration must be documented by the organization and available to the
registrar (certification/registration body) and customer on request.
d. Customers who are members of the QuEST Forum shall provide the
necessary TL 9000 field performance data to the suppliers in order to
calculate the specific measurements.
3.3 Principles of
Measurements
Usage
a.
b.
c.
d.
3.4.1
Aggregation of Products
3.4 Measurements
Data Aggregation
If an organization wishes to register multiple products in the same product
and Customer Base category and clearly identifies them as separate in the registration scope, the
organization may report the data for each product separately. Similarly, if an
organization registers a business unit or a location, the organization has the
option to determine which products will be registered and how the data will be
aggregated.
3.4.2
Customer Base
(1)
(2)
Total: All of the organizations customers for the product(s) to which the
measurement applies.
c. The organization shall report measurement data from only one customer
base per individual measurement.
3.5 Responsibilities
3.5.1
f.
(1)
(2)
(3)
g. Provide and control access to the measurement data output through the
QuEST Forum web site,
i.
j.
3.5.2
(3)
f.
g.
h.
i.
l.
m. Analyze research data to reveal industry trends and report only to the
measurements work groups, and
n. Analyze research data to determine if there are conditions under which the
data could be compared and make recommendations only to the
measurements work groups to achieve comparability.
3.5.3
Organization Responsibilities
b. Collect, validate, and submit data per the defined measurement definitions to
the Measurements Administrator using the provided tool(s),
f.
Provide measurement data for new products within six months from General
Availability of the product, if it falls within the scope of registration,
i.
j.
3.5.4
Customer Responsibilities
3.5.5
f.
3.5.6
Registrar Responsibilities
Section 4
4.1 Measurements
Listing
Handbook Section
5
5.1
5.2
6
6.1
7
7.1
8
8.1
8.1.5
9
9.1
5.3
5.4
8.1.6
8.1.7
4.2 Measurements
Reporting
Requirements
4.2.1
The supplier shall generate and distribute the measurement data to the
Measurements Administrator (and to customers according to the principles of
measurements processing as detailed in Section 3) as described by the profiles
in this handbook for the applicable product categories. The measurement data
shall conform to the requirements in the corresponding profile. Changes to
reported data that are needed to comply with a new master version of
Appendix A or a new version of the measurements handbook shall be completed
within six months of their release. All data reported commencing with the second
data submission after a new release of this handbook shall be in compliance with
the new release of the handbook.
4.2.2
For each product the supplier shall identify product categories and applicable
measurements according to Measurement Applicability Table (Normalized Units),
Appendix A, Table A-2. Appendix A is current as of the release of this handbook.
The Measurement Applicability Table is subject to periodic updates. To
accommodate these changes, the master is available on the Quest Forum web
site (http://www.questforum.org/). The master shall be used in conjunction with
registrations and for all data submittals to the QuEST Forum database.
4.3 Measurements
Data and Reports
4.3.1
When the customer does not provide data required for a measurement, the
supplier shall not be required to report the measurement for that customer.
Organizations shall submit data for a measurement if any of their customers
provide the information. Organizations are exempt from submitting measurement
data to the Measurements Administrator if none of their customers provide the
required information.
4.3.2
When the measurement profile states that RQMS (GR-929-CORE, Reliability and
Quality Measurements for Telecommunications Systems (RQMS) [1]) alternative
reporting is acceptable, under the Method of Delivery and Reporting topic in the
profile, the following shall apply:
(1)
The data used for reporting to the QuEST Forum and its members
include all applicable data as prescribed by the TL 9000 definition of
the measurement and it is not limited to the RQMS client company
subset of data.
(2)
4.3.3
Unless otherwise specified by the profile, the supplier shall collect data monthly
and report the required results quarterly to the QuEST Forum Measurements
Administrator. The supplier is free to use whatever time periods or formats
appropriate for reporting internally and to its customers. The quarterly update
shall include the data points from the preceding three months. Except for preregistration data submittals, all submissions shall be by calendar quarter.
4.3.4
The supplier shall report TL 9000 measurement data based on calendar months
or defined fiscal months. The supplier shall use the chosen method consistently.
The supplier shall notify customers and the Measurements Administrator prior to
changing methods. The supplier shall use calendar days for the measurements
that involve number of days.
4.3.5
The supplier shall report data for all applicable measurements defined in this
handbook to the Measurements Administrator according to the agreed rules.
This reporting requirement applies whether the supplier uses the TL 9000
method or the RQMS alternative reporting and whether the measurement
includes the designation compared data (CD) or research data (RD). See the
Measurements Summary Listing, Table A-5 in Appendix A.
NOTE: The designation compared data in the Method of Delivery and Reporting
section of the profile means that industry statistics may be available from the
QuEST Forum Administrator. However, the designation research data
indicates that no comparable industry statistics are available and the
Measurements Administrator will report analyses of industry trends only to the
QuEST Forum measurements work group.
4.3.6
Product Exclusions
The supplier may exclude data on products that are no longer supported for its
general customer base. This exclusion does not apply to individual field
replaceable units that have been made obsolete by a later version unless those
units are completely recalled from the field. Formal notification of placement of
the product on Additions and Maintenance (A&M) or Manufacturing
Discontinued (MD) status shall have been made to the customers for this
exclusion to apply.
4.3.7
Measurement Applicability
4.3.8
Where the normalization factor is traffic capacity based, such as DS1, OC-1, DSL
or Terminations, the calculation shall be based on the true useable traffic
capacity. Equipment within the system used to provide protection for the main
traffic path shall not be included, as it does not add useable capacity to the
system.
Section 5
Common Measurements
Common measurements are measurements that apply to all products: hardware,
software, and services.
5.1
5.1.1
5.1.2
Purpose
5.1.3
5.1.4
Detailed Description
a. Terminology
The Glossary includes definitions for the following terms used for the NPR
Measurement:
Problem Report
Service Problem Report
Severity Level
b. Counting Rules
The following rules shall apply in counting problem reports for the NPR
measurement.
(1)
(2)
(3)
(4)
(5)
(6)
Duplicate problem reports, i.e., the same fault has occurred either at a
different customer location or at another time, shall each be counted as
separate problem reports.
(7)
(8)
(9)
(11) NPR problem reports are counted in the month they are received and
only in the month they are received.
(1)
(2)
(3)
(4)
(2)
(3)
Notation
NU
S
Afactor
Np1
Np2
Np3
Np4
=
=
=
=
=
=
Table 5.1-1
Identifier
NPR1
NPR2
NPR3
NPR4
Title
H/S Critical Problem Reports
per NU per year
H/S Major Problem Reports
per NU per year
H/S Minor Problem Reports
per NU per year
Service Problem Reports
per NU per year
Table 5.1-2
Identifier
IPR1
IPR2
IPR3
Formula
Np1 x Afactor / S
Np2 x Afactor / S
Np3 x Afactor / S
Np4 x Afactor / S
Title
Incoming Critical Problem Reports per system per month
Incoming Major Problem Reports per system per month
Incoming Minor Problem Reports per system per month
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
TL 9000 NPR Data Table (Table 5.1-3) The NPR measurement shall
be reported with data elements (or equivalent as defined by the QuEST
Forum Administrator) for each month and each product category as
follows:
Table 5.1-3
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
NPR1 Numerator:
NPR2 Numerator:
NPR3 Numerator:
NPR4 Numerator:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability
Table (Normalized Units),
Appendix A, Table A-2
TL 9000
Either Total or Forum
S or none
Afactor (see Glossary)
NPR
Np1
Np2
Np3
Np4
(3)
Table 5.1-4
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Measurement Identifier:
IPR1 Numerator:
IPR2 Numerator:
IPR3 Numerator:
5.1.5
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability
Table (Normalized Units), Appendix A,
Table A-2
RQMS
Either Total or Forum
Number of systems in service
IPR
IPR1n Number of incoming critical problem
reports
IPR2n Number of incoming major problem
reports
IPR3n Number of incoming minor problem
reports
Sources of Data
a. Customers
b. Suppliers
5.1.6
CD
CD
CD
CD
5.1.7
YES
YES
YES
NO
Example Calculations
(2)
Table 5.1-5
Year:
Month
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
NPR1 Numerator: Np1
NPR2 Numerator: Np2
NPR3 Numerator: Np3
NPR4 Numerator: Np4
YYYY
MM
Provided by QuEST Forum Administrator
4.2
TL 9000
Total
30
12
NPR
0
3
45
NA
(3)
Table 5.1-6
Problem
Reports
Np1 = 0
Severity Afactor
Critical 12
Np2 = 3
Major
12
Np3 = 45
Minor
12
Np4 = NA
Normalization
Factor (S)
NPR Measurement Result
NPR1 = 0
30
Critical Problem
Reports per
system per year
NPR2 = 1.2 Major Problem
30
Reports per
system per year
NPR3 = 18 Minor Problem
30
Reports per
system per year
NPR4 = NA Service Problem
Reports are not
applicable for this
product
(2)
Table 5.1-7
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
NPR1 Numerator: Np1
NPR2 Numerator: Np2
NPR3 Numerator: Np3
NPR4 Numerator: Np4
YYYY
MM
Provided by QuEST Forum Administrator
7.3
TL 9000
Total
20
12
NPR
NA
NA
NA
30
(3)
Table 5.1-8
Problem
Reports Severity
Np1 = NA Critical
5.2
5.2.1
5.2.2
Purpose
5.2.3
5.2.4
Detailed Description
a. Terminology
The Glossary includes definitions for the following terms used for the FRT
Measurements:
Closure Criteria
Closure Date
Closure Interval
Fix
Fix Response Time
Official Fix
Overdue Service Problem Report
Problem - Critical H/S
Problem - Major H/S
Problem - Minor H/S
Problem Report
Severity Level
Temporary Fix
b. Counting Rules
(1)
Only Problem Reports that are originated by a customer and meet the
criteria for Number of Problem Reports shall be included. All counting
rules and exclusions noted in section 5.1.4.b and 5.1.4.c also apply to
FRT.
(2)
The start of the interval for calculating FRT shall be considered as the
receipt of that problem report by the supplier. If the severity of a
problem report is modified, the FRT shall start at the receipt of the
problem report.
(3)
The end of the interval for calculating FRT shall be considered as the
date that the official fix or closure criteria is made available. Should the
problem report originator later reject the fix as incomplete or causing
side effects, the problem report shall be re-classified as open.
(4)
For FRT, problem reports are counted ONLY in the month they are due
and not in the month they are fixed if different.
(5)
The total FRT shall be reported in the severity classification at the time
the fix is due to be closed.
(6)
(7)
(8)
If the deployment of the fix is delayed (or does not occur) specifically at
customer request (and not because of supplier problems), the interval
is defined as ending when the official fix is first made available for
delivery. The delay interval shall not be included in the FRT
calculation.
(9)
(2)
(3)
Notation
Fr2
Fr3
Fr4
Fr2d
Fr3d
Fr4d
=
=
=
=
=
=
Table 5.2-1
Identifier
FRT2
FRT3
FRT4
Title
H/S Major Problem Reports
Fix Response Time
H/S Minor Problem Reports
Fix Response Time
Service Problem Reports
Fix Response Time
Formula
(Fr2 / Fr2d) x 100
(Fr3 / Fr3d) x 100
(Fr4 / Fr4d) x 100
Note
% delivered
on time
% delivered
on time
% resolved
on time
Table 5.2-2
Identifier
ORT2
ORT3
Title
% Major Problems Closed On Time
% Minor Problems Closed On Time
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
Table 5.2-3
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
FRT2 Numerator:
FRT3 Numerator:
FRT4 Numerator:
FRT2 Denominator:
FRT3 Denominator:
FRT4 Denominator
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability
Table (Normalized Units), Appendix A,
Table A-2
TL 9000
Either Total or Forum
FRT
Fr2
Fr3
Fr4
Fr2d
Fr3d
Fr4d
(3)
Table 5.2-4
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
ORT2 Numerator:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability
Table (Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
ORT
Ort2n The total number of major fixes due to be
closed during the three-month window that were
delivered on time
ORT2 Denominator:
ORT3 Numerator:
ORT3 Denominator
5.2.5
Sources of Data
The data for the FRT measurement are derived from information provided by
customers and from supplier analysis as follows:
a. Customers
b. Suppliers
Track problem reports, their severity (H/S), the agreed closure interval
(services), and actual closure dates
Count due, overdue and on-time fixes and problem reports, and compute
the measurements according to the stated rules.
5.2.6
CD
CD
CD
5.2.7
YES
YES
NO
Example Calculations
(2)
Table 5.2-5
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
FRT2 Numerator: Fr2
FRT3 Numerator: Fr3
FRT4 Numerator: Fr4
FRT2 Denominator: Fr2d
FRT3 Denominator: Fr3d
FRT4 Denominator: Fr4d
(3)
YYYY
MM
Provided by QuEST Forum Administrator
4.2
TL 9000
Total
FRT
5
20
NA
5
25
NA
Table 5.2-6
Fixes
On-Time
Fr2 = 5
Severity
Major
Fixes Due
Fr2d = 5
FRT Measurement
Results
FRT2 = 100% Major H/S Problem
Report Fixes Delivered
On Time
Fr3 = 20
Minor
Fr3d = 25
FRT3 = 80%
Fr4 = NA
Services
Fr4d = NA
FRT4 = NA
Services Problem
Reports
are not applicable for
this product
(2)
Table 5.2-7
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
FRT2 Numerator: Fr2
FRT3 Numerator: Fr3
FRT4 Numerator: Fr4
FRT2 Denominator: Fr2d
FRT3 Denominator: Fr3d
FRT4 Denominator: Fr4d
(3)
YYYY
MM
Provided by QuEST Forum Administrator
7.1
TL 9000
Total
FRT
NA
NA
16
NA
NA
20
Table 5.2-8
On-Time
Closures
Fixes Due
FR4 = 16
Fr4d = 20
FRT4 = 80%
Service Problem
Reports Resolved
On Time
Event
Event Date
March 1
March 10
March 12
Problem Due
Date
March 31
March 31
In suspense
April 1
April 18
The effect of the site not being available is to move the due date of the problem
report from March 31 to April 18. The difference is the length of the delay. The
problem report would therefore be reported with the April data per counting rule
5.2.4 b. (4).
5.3
5.3.1
5.3.2
Purpose
5.3.3
5.3.4
Detailed Description
a. Terminology
The Glossary includes definitions for the following terms used for the OFR
Measurements:
Closure Criteria
Closure Date
Closure Interval
Fix
Fix Response Time
Official Fix
Overdue Service Problem Report
Problem - Critical H/S
Problem - Major H/S
Problem - Minor H/S
Problem Report
Severity Level
Temporary Fix
b. Counting Rules
In addition to the rules contained in section 5.2, the following rules shall apply.
(1)
Overdue problem reports are those that are open beyond the due
threshold time. The due threshold time is defined as:
30 calendar days for major H/S problem reports and
180 calendar days for minor H/S problem reports
A closure date agreement made between the customer and the
supplier for all service problem reports. Expected closure intervals
for services may be predetermined by a contractual agreement.
(2)
(3)
Penalty problem reports are counted in the OFR measurement and are
applicable only to hardware and software products. A penalty problem
report is defined as:
For majors, all problem reports with age since opening which
exceed 180 calendar days,
For minors, all problem reports with age since opening which
exceed 270 calendar days,
Penalty problem reports shall also be counted as overdue problem
reports (that is, double counting constitutes the penalty).
The sum of penalty problem reports for the month shall be added to
the sum of the overdue problem reports for the month.
The number of overdue problem reports closed are those overdue
problem reports that were closed in the month.
The measurement is computed as the number of overdue problem
reports closed divided by the sum of overdue problem reports and
the total number of penalty problem reports; the result shall be
expressed as a percentage.
The measurement shall be reported as 100% in the case where
there are no overdue problem reports during the period.
Notation
Pro2
Pro3
Pro4
Prp2
Prp3
Prc2
Prc3
Prc4
=
=
=
=
=
=
=
=
Table 5.3-1
Identifier
OFR2
OFR3
OFR4
Title
H/S Major Overdue Problem
Report Fix Responsiveness
H/S Minor Overdue Problem
Report Fix Responsiveness
Service Overdue Problem
Report Fix Responsiveness
Table 5.3-2
Identifier
OPR2
OPR3
Note
%
closed
%
closed
%
closed
Title
% Rate of Closures of Overdue Problem Reports Major
% Rate of Closures of Overdue Problem Reports Minor
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
Table 5.3-3
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OFR2 Numerator:
OFR3 Numerator:
OFR4 Numerator:
OFR2 Denominator:
OFR3 Denominator:
OFR4 Denominator
nd
OFR2 Denominator 2 Term:
nd
OFR3 Denominator 2 Term:
(3)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability
Table (Normalized Units), Appendix A,
Table A-2
TL 9000
Either Total or Forum
OFR
Prc2
Prc3
Prc4
Pro2
Pro3
Pro4
Prp2
Prp3
Table 5.3-4
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OPR2 Numerator:
OPR2 Denominator:
OPR3 Numerator:
OPR3 Denominator:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
OPR
Opr2n The sum of the overdue major problem
reports closed in the three-month period.
Opr2d The sum of penalty major problem
reports for the three-month period added to the
sum of the overdue major problem reports for the
same period
Opr3n The sum of the overdue minor problem
reports closed in the three-month period
Opr3d The sum of penalty minor problem
reports for the three-month period added to the
sum of the overdue minor problem reports for the
same period
5.3.5
Sources of Data
The data for the OFR measurement are derived from information provided by
customers and from supplier analysis as follows:
a. Customers
b. Suppliers
Track problem reports, their severity (H/S), the agreed closure interval
(services), and actual closure dates
Count due, overdue and on-time fixes and problem reports, and compute
the measurements according to the stated rules.
5.3.6
RD
RD
RD
5.3.7
YES
YES
NO
Examples
At the beginning of the month, there were six major H/S problem
reports that were overdue (age > 30 calendar days). One of these
became a penalty major H/S problem report during the month (age >
180 calendar days). Two of the six overdue reports were closed during
the month. There was no overdue minor H/S problem report at the
beginning of the month. However, by the end of the month five minor
H/S problem reports for which fixes had been due during the month
had become overdue. One of these overdue minor H/S problem
reports was closed before the end of the month.
(2)
Table 5.3-5
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OFR2 Numerator: Prc2
OFR3 Numerator: Prc3
OFR4 Numerator: Prc4
OFR2 Denominator: Pro2
OFR3 Denominator: Pro3
OFR4 Denominator: Pro4
nd
OFR2 Denominator 2 Factor: Prp2
nd
OFR3 Denominator 2 Factor: Prp3
(3)
YYYY
MM
Provided by QuEST Forum Administrator
4.2
TL 9000
Total
OFR
2
1
NA
6
5
NA
1
0
Table 5.3-6
Closed
Overdue
Fixes
Problems Severity Overdue
Prc2 = 2
Pro2 = 6
Major
Penalty
Problem
Reports
Prp2 = 1
At the beginning of the month, there were two Service problem reports
that were overdue (age greater than the agreed closure interval). One
of the two overdue reports was closed during the month.
(2)
Table 5.3-7
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OFR2 Numerator: Prc2
OFR3 Numerator: Prc3
OFR4 Numerator: Prc4
OFR2 Denominator: Pro2
OFR3 Denominator: Pro3
OFR4 Denominator: Pro4
nd
OFR2 Denominator 2 Factor: Prp2
nd
OFR3 Denominator 2 Factor: Prp3
(3)
YYYY
MM
Provided by QuEST Forum Administrator
7.1
TL 9000
Total
OFR
NA
NA
1
NA
NA
2
NA
NA
Table 5.3-8
Closed
Overdue
Problems
Prc4 = 1
Severity
not
applicable
Penalty
Problem
Reports
not
applicable
OFR
Measurement Result
OFR4 = 1 / 2 x 100 = 50%
% Overdue Service
Problem Reports Closed
5.4
5.4.1
5.4.2
Purpose
5.4.3
5.4.4
Detailed Description
a. Terminology
A service order is an order for service having a Customer Requested Completion
Date (CRCD), but not an installed system order. An example of a service order
is when a supplier is contracted by a customer to install and/or engineer a
product that is manufactured by another supplier. Services may include
engineering and/or installation.
The Glossary includes definitions for the following terms used for the OTD
measurement:
Installed System
Installed System Order
On-Time Installed System Delivery
On-Time Item(s) Delivery
b. Counting Rules
(1)
(2)
(3)
Due dates and delivery dates are considered to be one 24-hour period
(customers calendar day).
(4)
(5)
(6)
(7)
The monthly OTD data shall include all orders having CRD occurring
during the same month.
(8)
Actual On-Job Date (AOJD) identifies the date when the shipment
actually was delivered at the ship-to address. This date is derived by
adding the transportation interval to the actual ship date.
(9)
Late Orders Received (LOR) are those for which CRD is earlier than
Date Order Received, and are excluded from the measurement.
(2)
(3)
Notation
Cs
Ss
Ci
Si
Cv
Sv
=
=
=
=
=
=
Number of installed systems for which CRCD occurred during the month
Number of installed systems accepted on the CRCD during the month
Number of items for which CROJD occurred during the month
Number of items accepted on the CROJD during the month
Number of service orders for which CRCD occurred during the month
Number of service orders accepted on the CRCD during the month
Table 5.4-1
Identifier
OTIS
OTI
OTS
Title
On-time Installed
System Delivery
On-time Items
Delivery
On-time Service
Delivery
Formula
(Ss / Cs) x 100
(Si / Ci) x 100
(Sv / Cv) x 100
Note
% accepted
on CRD
% accepted
on CRD
% accepted
on CRD
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
Table 5.4-2
Year:
Month
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OTIS Numerator:
OTI Numerator:
OTS Numerator:
OTIS Denominator:
OTI Denominator:
OTS Denominator:
5.4.5
YYYY
MM
Provided by the QuEST Forum
Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table
A-2
TL 9000
Either Total or Forum
OTD
Ss
Si
Sv
Cs
Ci
Cv
Sources of Data
CD
CD
CD
5.4.7
Examples
Table 5.4-3
CRD
Purchase
Order
mm/dd
A
03/10
D
E
Number of
Orders
TOTALS:
March
5
OTD
(Ss/Cs)
03/20
03/21
02/15
03/25
Number of
System
CRDs Due in
Month (Cs)
4
Quantity
Ordered
5
6
4
8
12
1
2
1
2
1
2
2
7
1
1
Quantity
Installed
5
6
4
4
4
6
6
2
1
1
7
1
1
Date
Installed
3/10
3/10
3/10
3/22
3/23
3/22
3/25
3/21
3/21
3/22
3/15
3/15
3/25
Date
On-time
Note
Accepted Installations
1
3/10
0
3/25
0
3/22
3/15
4/15
NA
On-time
Purchase
Orders (Ss)
1
OTIS =
25.0%
NOTES:
1. Order B 2 line items were split into 4 partial installations each with a separate date
installed.
2. PO system D CRD was not counted in the total of 4 for March as it had a February CRD.
3. PO E Service Order while installed on time, did not meet customer acceptance until
supplier changes were completed and after the CRD and therefore was not on time.
4. The CRD installed system OTDI performance for March was 25% or 1(CRD met) / 4
(CRDs due).
5. It should be noted the line items and associated quantities are shown for completeness.
They have no direct impact in the calculation of OTD for installed systems other than the
system installation has not been completed until the last line item has been accepted.
Table 5.4-4
PO
CRD
mm/dd
3/10
3/20
3/21
2/15
1
2
3
1
5
6
4
8
12
1
2
1
2
1
2
2
7
1
1
5
6
4
4
4
6
6
2
1
1
7
1
1
3/10
3/10
3/10
3/22
3/23
3/22
3/25
3/21
3/21
3/22
3/15
3/15
3/15
OTS
CRD
mm/dd
1
3/10
0
3/25
0
3/22
3/15
3/25
NA
3/25
Number
of
Orders
CRDs
Due in
March
Cv
On-time
Orders SV
1
OTS =
25%
TOTAL:
5
March OTD
Sv/Cv
Note
NOTES:
1. PO system I CRD was not counted in the total of 3 for March as it had a
February CRD date and it was previously counted.
2. Service Order was completed but not accepted for early delivery. Thus, CRD
was not met.
3. The CRD OTD performance for March was 25% or 1(CRD met) / 4 (CRDs
due).
4. It should be noted the line items and associated quantities are shown for
completeness. The service has not been delivered until the last item has been
accepted.
Table 5.4-5
PO
K
M
N
O
Qty
CRD Split
Qty
Actual OTD Note
Line
Item Ordered
Order Delivered
CRD
1
5
3/10
5
3/10
1
2
6
3/12
6
3/13
0
3
4
3/17
4
3/18
0
1
8
3/20
8
3/22
0
2
12
3/22
y
6
3/22
0
1
y
6
3/25
0
1
3
2
3/29
2
?
0
2
4
2
3/30
2
3/30
1
1
7
2/15
7
3/15
NA
3
2
1
2/15
1
3/15
NA
3
1
20
3/25
y
10
3/25
0
4
y
10
3/25
1
4
1
2
3/10
2
3/5
0
5
Number Number
of
of Line
Orders Items
TOTAL:
March
9
OTD
Si/Ci
CRDs
Due in
March
Ci
On-time
Orders
Si
3
OTI =
33%
NOTES:
1. Line item L2 was not on time for CRD because only of the items were
delivered to CRD.
2. ? - OTD date could not be confirmed and therefore the line item is
assumed to have missed OTD.
3. PO line items M1 and M2 CRDs were not counted in the total of 9 for March
as they had Feb CRD dates and were previously counted.
4. Line item N1 is counted as 1 on time line item because while both portions of
the split shipments were delivered on time, it is still just 1 line item on the
order.
5. Line item O1 was delivered early. Thus, CRD was not met.
6. The CRD OTD performance for March was 33% or 3 (CRD met) / 9 (CRDs
due).
Table 5.4-6
Year
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Measurement Identifier:
OTIS Numerator: Ss
OTIS Denominator: Cs
OTI Numerator: Si
OTI Denominator: Ci
OTS Numerator: Sv
OTS Denominator: Cv
2000
03
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Total
OTD
1
4
3
9
1
4
Section 6
6.1
6.1.1
6.1.2
Purpose
6.1.3
6.1.4
Detailed Description
The supplier shall provide two sets of measurements for each product category
code: (1) overall outage frequency and downtime, and (2) supplier attributable
outage frequency and downtime.
a. Terminology
Downtime Performance Measurement (DPM):
DPM applies only to suppliers that elect RQMS equivalent reporting. DPM is the
expected long-term average sum, over one operating year, of the time duration of
events that prevent a user from requesting or receiving services. A failure that
causes service interruption contributes to the outage downtime of that service.
Outage downtime is usually expressed in terms of minutes per system per year.
Outage Frequency Measurement (OFM):
OFM applies only to suppliers that elect RQMS equivalent reporting. OFM is the
expected long-term average number of events, per unit time, that cause a loss of
services to the service provider Outage frequency is usually expressed in terms
of incidents per system per year.
The Glossary includes definitions for the following terms used for the SO
measurement:
Customer Base
Scheduled Outage
Total System Outage
b. Counting Rules
Unless an exclusion applies, all outages representing loss of functionality
shall be counted as follows:
(1)
(2)
(3)
(4)
(5)
For systems that are not continuously operational (24X7), count only
outages and duration that occur during the operational window.
(6)
(7)
Outages are counted in a product only when the failure is within the
product itself.
(8)
(1)
(2)
(3)
(4)
(5)
(2)
(3)
Notation
P
Ps
Q
Qs
S
=
=
=
=
=
Afactor
NU
=
=
Table 6.1-1
Identifier
SO1
SO2
SO3
SO4
Title
Annualized Weighted
Outage Frequency
Annualized Weighted
Downtime
Annualized Supplier
Attributable Outage
Frequency
Annualized Supplier
Attributable Downtime
Formula
Q x Afactor / S
P x Afactor / S
Qs x Afactor / S
Ps x Afactor / S
Note
Weighted outages
per NU per year
Minutes per NU
per year
Weighted outages
per NU per year
Minutes per NU
per year
Table 6.1-2
NOTE: Report only the measurements in this table that are applicable to the
specific product as defined by RQMS. [1]
Identifier
rDPMsn
hDPMsn
rDPMcn
hDPMcn
rOFMsn
hOFMsn
rOFMcn
hOFMcn
Title
Supplier Attributable Total Outage Minutes per System per Year
remote only
Supplier Attributable Total Outage Minutes per System per Year
host only
Service Provider Attributable Total Outage Minutes per System
per Year remote only
Service Provider Attributable Total Outage Minutes per System
per Year host only
Supplier Attributable Total Outages per System per Year
remotes
Supplier Attributable Total Outages per System per Year
hosts
Service Provider Attributable Total Outages per System per Year
remotes
Service Provider Attributable Total Outages per System per Year
hosts
Table 6.1-3
Identifier
DPMn
DPMsn
OFMn
OFMsn
Title
Total Outage Minutes Per System Per Year overall
Total Outage Minutes Per System Per Year supplier attributable
Total Outages Per Year overall
Total Outages Per Year supplier attributable
(4)
Notation
N
M
Pi
Ai
Sn
S
=
=
=
=
=
=
S = n =1 S n
N
Downtime Formulas
Systems of Uniform Size
For the special case of products of uniform system size where only total system
outages are possible, the downtime calculation is comparable to the current
RQMS calculation for end offices. Downtime is computed as follows for monthly
data:
m
DT = 12x
Pi
i=1
(6.1-1)
Examples include toll ticketing, voice messaging, SMDR, dispatch systems, etc.
All other Systems
Downtime for all other products (where systems consist of lines, ports,
terminations, or other normalization units) is computed as follows for monthly
data:
m
DT = 12 x
AP
i=1
N
S
n =1
(6.1-2)
OF = 12 x
m
N
(6.1-3)
Examples include toll ticketing, voice messaging, SMDR, dispatch systems, etc.
All other Systems
Outage Frequency for all other products (where systems consist of lines, ports,
terminations, or other normalization units) is computed as follows for monthly
data:
m
OF = 12 x
i=1
N
S
n=1
(6.1-4)
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
Table 6.1-4
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
P:
Ps:
Q:
Qf:
(3)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Either Total or Forum
S
Afactor (see Glossary)
SO
DT Calculated downtime in minutes/year for all
causes
DT Calculated downtime in minutes/year for all
supplier-attributable causes
OF Calculated outage frequency in
occurrences/year for all causes
OF Calculated outage frequency in
occurrences/year for supplier-attributable causes
Table 6.1-5
NOTE: If separation of host and remote systems does not apply, report all items
under the host category.
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Measurement Identifier:
rDPMsn:
hDPMsn:
rDPMcn:
hDPMcn:
rOFMsn:
hOFMsn:
rOFMcn:
hOFMcn:
Table 6.1-6
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
rS Total systems deployed per RQMS
remote only
hS Total systems deployed per RQMS host
only
SOE
Annualized supplier attributable total outage
minutes remote only
Annualized supplier attributable total outage
minutes host only
Annualized service provider attributable total
outage minutes remote only
Annualized service provider attributable total
outage minutes host only
Annualized supplier attributable total outage
frequency remote only
Annualized supplier attributable total outage
frequency host only
Annualized service provider attributable total
outage frequency remote only
Annualized service provider attributable total
outage frequency host only
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Measurement Identifier:
DPMn:
DPMsn:
OFMn:
OFMsn:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
S Total systems deployed per RQMS
SOG
Annualized total outage minutes for all causes
Annualized supplier attributable outage minutes
Annualized total outage frequency for all causes
Annualized supplier attributable outage minutes
6.1.5
Sources of Data
Customers shall report all outage data and system population of their end users
to the supplier. If outage data is not supplied, then the supplier is not responsible
for reporting this measurement.
6.1.6
CD
CD
CD
CD
6.1.7
YES
YES
YES
YES
Example Calculations
DT = 12x
Pi
i=1
OF = 12 x
DT = 12 x
AP
i
i=1
N
S
n =1
OF = 12 x
i=1
N
S
n=1
Table 6.1-7
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
P:
Ps:
Q:
Qs:
2001
3
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Forum
4
12
SO
90.0
90.0
8.625
8.625
Table 6.1-8
Outage
Length
(minutes)
Signal Signal
DS1
Type Quantity Equivalents
Number
of
Outages
(Ai)
(Pi)
60
3
16
5
DS3
1
1 x 28
DS1
8
8 x1
DS1
1
1x1
OC-3
1
1 x 84
Total month m, Q, P
Annualized
1
1
1
1
M=4
48
Weighted
Frequency
(DS1 x
number)
Q=Si
28
8
1
84
Q = 121*
1452
Weighted
Time
(DS1 x
minutes)
(Ai x Pi)
1680
24
16
420
P = 2140*
25680
*reported items
Dividing the annualized totals by the 32,000 DS1 capacity (S = 32,000 DS1
Equivalents), the normalized downtime numbers are:
Table 6.1-9
Digital Cross-connect
Downtime
(min / equivalent DS1 / yr)
Outage Frequency
(Count / equivalent DS1 / yr)
Overall
0.8025
Supplier
Attributable
0.8025
Unweighted
0.045
0.045
0.0015
6.1.8
2001
3
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Forum
32000
12
SO
0.8025
0.8025
0.045
0.045
Reliability Conversions
NOTE: The following analysis provides formulas to convert outage frequency and
downtime to other reliability measurements for reference. Equation 6.1-5
provides conversion from outage frequency to mean time between failures
(MTBF). Equation 6.1-6 provides conversion from downtime (expressed in
minutes) to system availability. System availability / unavailability and MTBF are
alternative expressions of system reliability found in some requirement
specifications.
(6.1-5)
This calculation represents the mean (average) number of hours between system
outages.
A = Availability (Reference only)
A = Probability that the system is operational when required
A = Up time Total time
A = {(365)(24)(60) DT} {(365)(24)(60)}
A = {525,600 DT} 525,600
(6.1-6)
A TSP = A X PX + A Y PY + A Z PZ
(6.1-8)
Section 7
Hardware Measurements
7.1
7.1.1
Initial Return Rate (IRR) return rate of units during the first six months after
initial shipment (months zero through six of shipment),
One-Year Return Rate (YRR) - return rate of units during the first year
following the Initial Return Rate period (months seven through 18 of
shipment),
Long-Term Return Rate (LTR) - return rate of units any time following the
One-Year Return Rate period (months 19 and later after shipment), and
7.1.2
Purpose
7.1.3
d. These measurements are not intended for items shipped in bulk such as:
Cable
Optical fiber
NOTE: The Initial Return Rate measurement for items warehoused outside of the
suppliers control, for an extended period before placement in service, may not
accurately reflect the actual return rate for product in service. This may also be
true of items sold through distributors.
NOTE: Long-Term Return Rates may become inaccurate for older products as
units are taken out of service.
NOTE: The return rate for low cost items after the expiration of any warranty
period is likely to be inaccurate if purchasing a new item is no more expensive
than repairing the failed one.
7.1.4
Detailed Descriptions
a. Terminology
The Glossary includes definitions for the following terms used for this
measurement:
Return
b. Counting Rules
The following rules shall apply when counting returns and shipments for the
return rate measurements.
(1)
(2)
(3)
The supplier shall document, for the specific measurement, the method
of determining which of the returns are from the corresponding basis
shipping period. This may be determined by any of the following
methods:
(4)
Units that fail due to the problem corrected by a recall before they can
be rotated are to be counted as returns.
(5)
(6)
(1)
(2)
(3)
(4)
(2)
(3)
(4)
Initial Return Rate (IRR) The Initial Return Rate measures the rate of
return of product during the reporting period from the population of
units shipped during the prior month through six months prior to the
reporting period. This basis shipping period is assumed to represent
the initial return rate of the product during installation, turn-up, and
testing. Returns from units shipped during the current month are also
included.
(5)
(6)
(7)
Notation
NU
Afactor
Ri
Ry
Rt
Si
Sy
St
=
=
=
=
=
=
Table 7.1-1
Identifier
IRR
YRR
LTR
NYR
Title
Initial
Return Rate
One-Year
Return Rate
Long-Term
Return Rate
Normalized OneYear Return Rate
Formula
(Ri / Si) x Afactor x 100
Note
% per year
% per year
% per year
(Ry / S) x Afactor
Returns per
NU
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
Data shall be reported for IRR, YRR and LTR. Compared data industry
statistics are based only on the normalized YRR measurement.
(3)
TL 9000 Return Rate Data Table - The return rates shall be reported with
data elements (or equivalent as defined by the Measurements
Administrator) for each month and each product category as follows:
Table 7.1-2
Year:
Month:
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
IRR Numerator:
YRR Numerator:
LTR Numerator:
IRR Denominator:
YRR Denominator:
LTR Denominator:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Either Total or Forum
S (value for computing NYR)
Afactor (see Glossary)
RR
Ri
Ry
Rt
Si
Sy
St
7.1.5
Source(s) of Data
The supplier should have available, as a part of its data systems the information
listed above needed to calculate these measurements. This includes:
b. FRU returns records The suppliers return records shall include the
identifier necessary to match returns with shipment records.
c. Third party returns records Units returned to a third party repair agency by
the customer or repaired by the customer itself shall be included in the return
counts when available. To have accurate measurements, it is necessary for
the customer to make it a contractual requirement of their third party repair
agencies to supply this data to the original equipment manufacturers.
7.1.6
At present, only NYR is considered to have compared data. The IRR, YRR, and
LTR data shall also be reported to the Measurements Administrator for future
use.
RD
RD
RD
CD
7.1.7
Example Calculations
Initial Returns: From units shipped in the period from July 1, 1998,
through January 31, 1999.
One-Year Returns: From units shipped in the period from July 1, 1997
through June 30, 1998.
JAN
FEB
MAR
One-Year Returns
1996
DEC
JAN
FEB
Long-Term Returns
OCT
NOV
DEC
1999
JAN
1997
JUL
AUG
SEP
One-Year Returns
OCT
NOV
DEC
APR
MAY
JUN
JUL
AUG
Initial Returns
MAR
APR
MAY
JUN
1998
SEP
Table 7.1-3
Example Returns
Ship
Jun-97 Jul-97 Aug-97 Sep-97 Oct-97 Nov-97 Dec-97 Jan-98 Feb-98 Mar-98 Apr-98 May-98 Jun-98
&
Date
before
Return
Mo.
Jan-99
39
22
11
17
19
16
24
11
7
14
10
6
6
Feb-99
44
9
11
16
13
8
16
11
9
15
9
11
13
Mar-99
42
11
14
17
21
15
17
7
8
12
14
12
12
Apr-99
46
12
12
12
15
14
22
9
11
8
10
11
16
May-99
31
11
19
16
17
21
12
9
10
10
9
16
11
Jun-99
35
10
15
16
11
28
19
9
8
15
8
9
7
Jul-99
48
7
13
17
14
17
17
9
5
4
10
13
9
Aug-99
36
10
1
7
19
17
15
13
5
12
6
16
12
Sep-99
46
8
16
16
16
19
24
6
3
7
12
8
14
Oct-99
41
15
10
11
18
13
14
3
14
9
11
13
13
Nov-99
32
16
13
12
17
14
15
6
7
5
11
10
7
Dec-99
30
5
21
17
13
20
14
3
9
12
10
3
13
Shipments:
30000
8253
9243
9261
9721
10131
10140
6263
6436
7244
7275
7396
8263
8833
8954
9368
9818
9787
10528
10644
11321
11332
11674
12151
12460
The annualized return rates for the month of January 1999, are calculated as:
Initial
(Returns of units shipped Jul-98 through Jan-99) x 12 x 100
Return Rate =
Total Shipments for Jul-98 through Dec-98
=
(14+16+20+39+36+23+5) x 12 x 100 _
8833+8954+9368+9818+9787+10528
= 3.20%
13494
During January 1999, the number of returned units was calculated as follows:
14 returns of units shipped in July 1998,
16 returns of units shipped in August 1998,
20 returns of units shipped in September 1998, and so on, including
5 returns of units shipped in the month of January 1999,
for a total number of initial returns of 153.
The corresponding field population is determined by the sum of the shipment
quantities shown in the bottom row of Table 7.1-3 for the months of July 1998
through December 1998. Note that the returns of units shipped in January are
included in order to count all returns during the month, and to be alerted to any
developing problems. However, shipments during January are excluded
because most units will not have been placed into operation.
One-Year
(Returns of units shipped Jul-97 through Jun-98) x 12 x 100
Return Rate =
Total Shipments for Jul-97 through Jun-98
=
(22+11+17+19+16+24+11+7+14+10+6+6) x 12 x 100
_
(8253+9243+9261+9721+10131+10140+6263+6436+7244+7275+7396+8263)
= 1.96%
Long-Term
(Returns from shipments prior to Jul-97) x 12 x 100
Return Rate =
Total Shipments prior to Jul-97
= 39 x 12 x100
30000
= 1.56%
Calculating the return rates for all months in 1999 gives:
Initial
Return
Rate
Jan-99
Feb-99
Mar-99
Apr-99
May-99
Jun-99
Jul-99
Aug-99
Sep-99
Oct-99
Nov-99
Dec-99
3.20%
2.80%
2.66%
2.44%
2.74%
2.57%
2.69%
2.80%
2.47%
2.39%
2.39%
2.14%
OneYear
Return
Rate
1.96%
1.72%
1.96%
1.96%
1.86%
1.65%
1.50%
1.81%
1.55%
1.55%
1.73%
1.57%
LongTerm
Return
Rate
1.56%
1.66%
1.69%
1.73%
1.70%
1.80%
1.84%
1.52%
1.86%
1.66%
1.56%
1.55%
ii.
50
100,000
40
100,000
5
50,000
3
10,000
v.
0
500
The normalizing factor for xDSL products is the number of DSL lines
deployed. Since one HTU-C and one HTU-R are required to deploy a
single HDSL line, the total number of lines deployed in the basis period is
100,000.
The normalized One-Year Return Rate would be:
[(50 + 40 + 5 + 3 + 0) x 12] / 100,000 = 0.012 returns / yr. / DSL line
(2)
ii.
10
1,000
5
500
2
500
250
(3)
Table 7.1-4
Year:
Month
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Annualization Factor:
Measurement Identifier:
IRR Numerator: Ri
YRR Numerator: Ry
LTR Numerator: Rt
IRR Denominator: Si
YRR Denominator: Sy
LTR Denominator: St
1999
01
Provided by QuEST Forum Administrator
1.1
TL 9000
Total
22000
12
RR
14
19
30
1200
1752
4500
Section 8
Software Measurements
8.1
8.1.1
8.1.2
Purpose
8.1.3
8.1.4
Detailed Description
a. Terminology
The Glossary includes definitions for the following terms used for software
update measurements:
General Availability
Patch
Patch Defective Corrective
Patch Defective Feature
Patch Official
Release Application
b. Measurement Options:
The measurement options outlined below group the installation and maintenance
activities together. In this way, the relationship between the installation and
maintenance efforts is apparent. Suppliers providing data shall submit their
results in accordance with the measurements contained in the selected option.
The selection criteria for the three options are described as follows:
Maintenance
S/W Update
Option 1
Option 2
S/W Update
S/W Update
Option 3
8.1.5
8.1.5.2 Purpose
This measurement is used to evaluate the percentage of release applications
with aborts with a goal of minimizing the service provider risk of aborts when
applying a software release.
General Availability
b. Counting Rules
(1)
(2)
(3)
(4)
(5)
(6)
The data shall include the three most dominant releases for each
product being reported. If fewer than three releases exist, the data
shall include all existing releases.
(2)
For each of the three most dominant releases, the supplier shall
provide the number of release application attempts for the month and
the number of systems that encountered any abort during the release
application/installation interval. The supplier shall report this data
monthly.
(3)
(4)
Notation
Release N
Release N-1
Release N-2
Ra0
=
Ar0
Ra1
Ar1
Ra2
Ar2
Title
Release Application
Aborts Release N
Release Application
Aborts Release N-1
Release Application
Aborts Release N-2
Formula
(Ar0 / Ra0) x 100
(Ar1 / Ra1) x 100
(Ar2 / Ra2) x 100
Note
% of systems
with aborts
% of systems
with aborts
% of systems
with aborts
Title
Cumulative % of Systems Experiencing an Abort during Release
Application Release N
Cumulative % of Systems Experiencing an Abort during Release
Application Release N-1
Cumulative % of Systems Experiencing an Abort during Release
Application Release N-2
Cumulative Number of Release Application Attempts
Release N
Cumulative Number of Release Application Attempts
Release N-1
Cumulative Number of Release Application Attempts
Release N-2
Data shall be reported quarterly. Each report shall include data for the
three months in the quarter.
(2)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Either Total or Forum
RAA
Ar0
Ar1
Ar2
Ra0
Ra1
Ra2
(3)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
RAQ
Cumulative number of systems experiencing an
abort during release application for Release N
Cumulative number of systems experiencing an
abort during release application for Release N-1
Cumulative number of systems experiencing an
abort during release application for Release N-2
Cumulative number of release applications for
Release N (Rar0)
Cumulative number of release applications for
Release N-1 (Rar1)
Cumulative number of release application
attempts for Release N-2 (Rar2)
CD
YES
1
Dec
2000
2
3
Jan Feb
2001 2001
4
Mar
2001
5
6
7
Apr May Jun
2001 2001 2001
8
Jul
2001
9
Aug
2001
10
Sep
2001
11
12
Oct Nov
2001 2001
13
Dec
2001
22
20
20
39
39
40
45
46
45
52
51
46
54
52
45
50
45
43
47
48
44
36
33
30
30
29
24
0
1
1
1
1
1
0
1
1
1
1
1
0
1
1
1
0
1
0
0
1
0
1
0
25
21
18
47
41
38
86
80
78
131
126
123
183
177
169
237
229
214
287
274
257
334
322
301
370
355
331
400
384
355
0
1
1
1
1
1
1
2
2
2
3
3
2
4
4
3
5
5
3
6
6
4
6
7
4
6
8
4
7
8
1.53
2.38
2.44
1.09
2.26
2.37
1.27
2.18
2.34
1.05
2.19
2.33
1.20
1.86
2.32
1.08
1.69
2.41
1.00
1.82
2.25
b. For the month of December 2001, the TL 9000 data reported for the above
example is shown in Table 8.1.5-6.
Measurement Methodology:
Customer Base:
Measurement Identifier:
RAA0 Numerator: Ar0
RAA1 Numerator: Ar1
RAA2 Numerator: Ar2
RAA0 Denominator: Ra0
RAA1 Denominator: Ra1
RAA2 Denominator: Ra2
2000
12
Provided by QuEST Forum
Administrator
From Measurement Applicability
Table (Normalized Units), Appendix
A, Table A-2
TL 9000
Forum
RAA
4
7
8
400
384
355
8.1.6
8.1.6.2 Purpose
This measurement is used to evaluate the percentage of defective official
patches with a goal of minimizing service provider risk of failure.
General Availability
Official Patch
Patch
b. Counting Rules
(1)
(2)
(3)
(4)
(5)
(6)
A defective patch shall be counted against the month during which the
patch was found defective.
(7)
The data shall include the three most dominant releases for each
product being reported. If fewer than three releases exist, the data
shall include all existing releases.
(2)
For CPQ, the supplier shall provide, by release, the total monthly
number of official corrective patches delivered and the number of
official corrective patches identified as defective.
(3)
For FPQ, the supplier shall provide, by release, the total monthly
number of official feature patches delivered and the number of official
feature patches identified as defective.
(4)
Notation
Release N
Release N-1
Release N-2
DPc0
=
DPc1
DPc2
Pc0
Pc1
Pc2
DPf0
DPf1
DPf2
Pf0
Pf1
Pf2
Title
Defective Corrective
Patches Release N
Defective Corrective
Patches Release N-1
Defective Corrective
Patches Release N-2
Defective Feature Patches
Release N
Defective Feature Patches
Release N-1
Defective Feature Patches
Release N-2
Formula
(DPc0 / Pc0) x 100
(DPc1 / Pc1) x 100
(DPc2 / Pc2) x 100
(DPf0 / Pf0) x 100
(DPf1 / Pf1) x 100
(DPf2 / Pf2) x 100
Note
% defective
per month
% defective
per month
% defective
per month
% defective
per month
% defective
per month
% defective
per month
Title
Monthly Number of Defective Corrective Patches Identified
Release N
Monthly Number of Defective Corrective Patches Identified
Release N-1
Monthly Number of Defective Corrective Patches Identified
Release N-2
Monthly Number of Defective Feature Patches Identified
Release N
Monthly Number of Defective Feature Patches Identified
Release N-1
Monthly Number of Defective Feature Patches Identified
Release N-2
Monthly Number of Corrective Patches Delivered Release N
Monthly Number of Corrective Patches Delivered Release N-1
Monthly Number of Corrective Patches Delivered Release N-2
Monthly Number of Feature Patches Delivered Release N
Monthly Number of Feature Patches Delivered Release N-1
Monthly Number of Feature Patches Delivered Release N-2
(2)
TL 9000 Data CPQ or FPQ Table The CPQ and FPQ measurements
shall be reported with data elements (or equivalent as defined by the
Measurements Administrator) for each month and each product
category combination as follows:
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Either Total or Forum
CPQ or FPQ
DPc0 (for CQP) or DPf0 (for FPQ)
DPc1 (for CQP) or DPf1 (for FPQ)
DPc2 (for CQP) or DPf2 (for FPQ)
Pc0
(for CQP) or Pf0
(for FPQ)
Pc1
(for CQP) or Pf1
(for FPQ)
Pc2
(for CQP) or Pf2
(for FPQ)
(3)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
DCP or DFP
Number of defective (corrective / feature)
patches for release N
Number of defective (corrective / feature)
patches for release N-1
Number of defective (corrective / feature)
patches for release N-2
Number of (corrective / feature) patches
delivered for release N (CPr0 or FPr0)
Number of (corrective / feature) patches
delivered for release N-1 (CPr1 or FPr1)
Number of (corrective / feature) patches
delivered for release N-2 (CPr2 or FPr2)
CD
RD
YES
YES
1
2
3
4
5
6
7
8
9
10
11
12
13
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan
2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2001
Number of Corrective
Patches Issued In Month
Pc0
Release N
52
Pc1
Release N-1
55
Pc2
Release N-2
60
53
55
55
48
50
50
35
40
47
34
36
42
34
32
35
32
34
35
30
36
31
28
33
32
30
32
30
25
26
29
24
24
27
22
24
25
0
1
2
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
1
0
1
1
1
0
0
0
1
0
0
0
0
2.94 0
0
0
2.94 0
0
0
3.23
0
0
0
0
0
4.17 0
0 3.85 0
4.17
3.33 3.45 0
0
b. For the month of November 2000, the TL 9000 CPQ data reported is shown
in Table 8.1.6-12.
2000
11
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Forum
CPQ
0
1
1
25
26
29
8.1.7
8.1.7.2 Purpose
This measurement is used to evaluate the level of defective software updates
with a goal of minimizing associated customer risks.
General Availability
b. Counting Rules
Software Updates
The following rules shall apply to the Software Update measurements:
(1)
(2)
(3)
The data shall include the three most dominant releases for each
product being reported. If fewer than three releases exist, the data
shall include all existing releases.
(1)
(2)
(3)
(1)
(2)
The data shall include the three most dominant releases for each
product being reported. If fewer than three releases exist, the data
shall include all existing releases.
(3)
(2)
(3)
(4)
(5)
Notation
Release N
Release N-1
Release N-2
Du0
=
Du1
=
Du2
=
Us0
=
Us1
=
Us2
=
Title
Defective Software Updates
Release N
Defective Software Updates
Release N-1
Defective Software Updates
Release N-2
Formula
(Du0 / Us0) x 100
(Du1 / Us1) x 100
(Du2 / Us2) x 100
Note
Cumulative
% defective
Cumulative
% defective
Cumulative
% defective
Title
Cumulative Number of Defective Software Updates
Release N
Cumulative Number of Defective Software Updates
Release N-1
Cumulative Number of Defective Software Updates
Release N-2
(2)
(3)
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
TL 9000
Either Total or Forum
SWU
Du0
Du1
Du2
Us0
Us1
Us2
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
RQMS
Either Total or Forum
DSU
Cumulative number of defective software
updates for release N
Cumulative number of defective software
updates for release N-1
Cumulative number of defective software
updates for release N-2
CD
YES
Jan Feb
2000 2000
11
1
2
0
9.1%
15
1
4
0
6.7%
Mar
2000
Apr
2000
May
2000
Jun
2000
25
2
25
2
8%
55
4
30
2
7.3%
10
2
10
2
20%
15
3
5
1
20%
30
4
15
1
13.3%
50
6
20
2
12%
18
1
3
0
5.6%
25
2
7
1
8.0%
28
4
3
2
14.3%
29
4
1
0
13.8%
2000
6
Provided by QuEST Forum
Administrator
2.1
TL 9000
Forum
SWU
4
6
4
55
50
29
Section 9
Services Measurements
9.1
9.1.1
9.1.2
Purpose
9.1.3
9.1.4
Detailed Description
a. Terminology
The Glossary includes definitions for the following terms used for these
measurements:
b. Counting Rules
(1) Failure of any unit during the repair warranty period shall be counted as
a defective repair unit.
(2)
(3)
(2)
Table 9.1-1
Service Category
Installation
Service Volume
(Abbreviation)
Total number of
installation and/or
engineering audits
% SQ
% SQ
(number of audits)
Maintenance
Number of
maintenance
callbacks
Total number of
maintenance visits
% SQ
(number visits)
Repair
Number of
defective repair
warranty units
Total number of
repaired units
% SQ
(number of repairs)
Customer Support
Service
Support Service
Number of
resolutions
exceeding
agreed time
Number of
defects
% SQ
(number of calls)
Total number of
transactions
% SQ
(number of
transactions)
NOTE: Service volume is a measure of the amount of service delivered.
NOTE: A nonconforming audit is one that fails to satisfy specified
acceptance requirements.
=
=
=
=
=
=
=
=
=
=
Sd5
Table 9.1-2
Identifier
Title
Formula
Note
SQ1
Conforming Installation
and/or Engineering
Audits
Successful Maintenance
Visits
(1-Sd1/S1) x 100
% audits
conforming
(1-Sd2/S2) x 100
SQ3
Successful Repairs
(1-Sd3/S3) x 100
SQ4
Conforming Customer
Support Service
Resolutions
Conforming Support
Service Transactions
(1-Sd4/S4) x 100
% visits without
maintenance
callbacks
% successful
repairs
% calls resolved
within agreed
time
% transactions
without Defect
SQ2
SQ5
(1-Sd5/S5) x 100
(2)
Table 9.1-3
YYYY
MM
Provided by QuEST Forum Administrator
From Measurement Applicability Table
(Normalized Units), Appendix A, Table A-2
Either Total or Forum
S1, S2, S3, S4, or S5
(as appropriate for the specific measurement
reported)
SQ1, SQ2, SQ3, SQ4, or SQ5
Sd1, Sd2, Sd3, Sd4, or Sd5
(as appropriate)
Customer Base:
Normalization Factor:
Measurement Identifier:
SQ Numerator:
9.1.5
Sources of Data
Table 9.1-4
Category
Installation
and/or
Engineering
Audits
Maintenance
Repair
Customer
Support
Service
Support
Service
SQ Data Sources
Source of Data
Supplier to count number of non-conforming supplier
installation and/or engineering and audits
9.1.6
RD
CD
CD
CD
RD
9.1.7
Example Calculations
a. Example 1 Installation
(1)
Table 9.1-5
Number of Non-conforming
Installation and/or Engineering Audits
Total Number of Installation and/or
Engineering Audits
Service Quality Measurement
January
5
February
1
March
0
April
6
100
50
75
80
95%
98%
100%
92.5%
Table 9.1-6
Year:
Month
Reporting ID:
Product Category Code:
Measurement Methodology:
Customer Base:
Normalization Factor:
Measurement Identifier:
SQ Numerator:
2000
01
Provided by QuEST Forum
Administrator
7.1
TL 9000
Forum
100
SQ1
5
b. Example 2 Repair
(1)
Table 9.1-7
January
2
February
0
March
1
April
4
30
20
75
120
93.3%
100%
98.6%
96.6%
(2)
(3)
c. Example 3 Maintenance
(1)
Table 9.1-8
(2)
January
2
30
93.3%
February
0
20
100%
March
1
75
98.6%
April
4
120
96.6%
(3)
Table 9.1-9
January
15
February
40
March
10
April
4
2000
5000
2750
3000
99.25%
99.2%
99.6%
99.8%
(2)
(3)
(1)
Cut Cables
(Number of Defects)
Number of Cables Identified
(Number of Opportunities for
Defects)
Service Quality Conformance
Measurement
(2)
January
5
February
2
March
0
April
4
1000
500
750
300
99.5%
99.6%
100%
98.6%
(3)
Appendix A
Appendix A
1.
List of Tables
2.
2.1
Appendix A
2.2
f.
g. The placement of the product in the hierarchy will reflect the dominant use of
the product.
Appendix A
Category:
Definition:
Switching
1.1
Circuit Switch
1.2
Note 1
Note 2
Note 3
Packet Switch
Examples:
End-office
Tandem
Tandem access
Remote
Service Switching Point
[SSP]
Mobile Switching Center
[MSC]
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-3
Appendix A
Category:
Definition:
Examples:
1.2.2
IP Packet Switch /
Router
1.2.3
Asynchronous Transfer
Mode (ATM) Switch
1.2.4
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-4
Appendix A
Category:
Definition:
Signaling
2.1
Note 1
Note 2
Note 3
Examples:
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-5
Appendix A
Category:
Definition:
Signaling Transfer
Point (STP)
2.3
Home Location
Register (HLR)
Transmission
3.1
Outside Plant
Note 1
Note 2
Note 3
Examples:
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-6
Appendix A
Category:
Definition:
Transmission
Medium
Metallic Products
3.1.1.1.1
Metallic Conductor
Cable
3.1.1.1.2
Metallic Connectors
3.1.1.2
Optical Fiber
Products
3.1.1.2.1
3.1.1.1
3.1.1.2.2
Optical Connectors
Examples:
Note 1
Note 2
Note 3
Metallic cable
Central office coaxial cable
Hybrid coaxial/twisted pair
drop
Coaxial connectors
Coaxial distribution
connectors
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-7
Appendix A
Category:
Definition:
Transmission
Sub-systems
3.1.1.3.1
Active Sub-systems
3.1.1.3.2
3.1.1.3.3
Examples:
Ancillary Sub-Systems
3.1.2
Physical
Structure
3.1.2.1
Enclosures
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-8
Appendix A
Category:
Definition:
Examples:
Support structures
3.1.2.3
Conduits
3.2
3.2.1
3.2.1.1
Note 1
Note 2
Note 3
Telephone poles
Pedestals
Microwave / radio towers
Innerduct
Multi-bore conduit
PVC pipe
Transport
Equipment
Appendix A
Category:
Definition:
Examples:
3.2.2
Carrier Systems /
Multiplexers
3.2.2.1
Interoffice / Long
Haul
3.2.2.1.1
3.2.2.1.2
Optical Carrier
System
3.2.2.1.2.1
SONET / SDH
Transport Systems
Digital Cross-connect
System (DCS)
Electronic DSX
Active Optical DSX
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-10
Appendix A
Category:
Definition:
Examples:
WDM / DWDM /
Optical Amplification
Products
3.2.2.1.3
Microwave
3.2.2.2
Loop Carrier
Wavelength Division
Multiplexer [WDM]
Dense Wavelength Division
Multiplexer
6, 8, 11, or 18 gigahertz
microwave radio
Digital loop carrier (DLC)
Universal digital loop carrier
(UDLC)
SLC remote terminal
Integrated digital loop
carrier
Analog loop carrier
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-11
Appendix A
Category:
Definition:
Examples:
Line Terminating
Equipment /
Distributing Frames
Equipment to provide the termination point for voice-grade and voicegrade compatible facilities and equipment in a central office. It is
composed of protectors, connectors and terminal strips or blocks.
Distributing frames are categorized as either conventional or modular.
3.2.4
3.3
Wireless
Transmission
Note 1
Note 2
Note 3
Tall conventional
distributing frames
Low-Profile Conventional
Distribution Frames
(LPCDFs)
Conventional protector
frames
Combined Main Distributing
Frame (CMDF)
Subscriber Main Distributing
Frame (SMDF)
Trunk Main Distributing
Frame (TMDF)
Intermediate Distributing
Frame (IDF)
Tie-Pair Distributing Frame
(TPDF).
Office repeater bays
ISDN
HDSL
ADSL
DDS
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-12
Appendix A
Category:
Definition:
Examples:
Base Station
Equipment
BSC
BSS
BTS
Wireless Repeaters
3.3.2
Base Transceiver
System (BTS)
3.3.3
Operations &
Maintenance
4.1
Test Systems
4.1.1
Test Access
Equipment
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-13
Appendix A
Category:
Definition:
Examples:
Test Equipment,
Embedded
4.1.3
4.2
Operations
Support Systems
4.2.1
On Line Critical
4.2.2
On Line Non-critical
4.2.3
Off Line
Traditional business systems that are run off line sometimes in batch
mode, typically overnight and do not have high availability expectations.
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-14
Appendix A
Category:
Definition:
Examples:
Ancillary Operations
and Maintenance
Products
Optical splicers
Single fiber fusion splicers
Mass fiber fusion splicers
Mechanical splicers
Portable test equipment
Optical connector tools
Cleavers
Common
Systems
5.1
Synchronization
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-15
Appendix A
5.3
Note 1
Note 2
Note 3
Category:
Definition:
Examples:
General Purpose
Computers
Terminals
PCs
Workstations
Mini, mid, mainframes
AC rectifiers/battery
chargers
Battery systems
Uninterruptible Power
Supplies (UPS)
DC to AC inverters
DC to DC bulk converters
AC and DC switch gear
Ring generator
Power distribution panels
Power Systems
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-16
Appendix A
Category:
Definition:
Customer
Premises
6.1
Enhanced
Services
Platforms
6.1.1
Interactive Voice
Response (IVR)
Platforms
Messaging Platforms
Multi-Application
Platforms
6.1.2
6.1.3
Note 1
Note 2
Note 3
Examples:
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-17
Appendix A
Category:
Definition:
Terminal
Equipment
6.2.1
Voice Terminals
6.2.1.1
Wireline Telephone
Sets
Wireless Subscriber
User Terminals
6.2.1.2
Examples:
The subscriber user terminal made to transmit and receive voice and/or
data communication using Telecommunication Infrastructure equipment
not requiring hard lines as a means of transport. User terminals may
be of any functional technology available for public use.
6.2.2
Fax equipment
6.2.3
6.2.4
Data Modems
Digital Data Service
Units
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-18
Appendix A
6.4
Category:
Definition:
Automatic Call
Distribution (ACD)
systems
Private Branch
Exchange (PBX)
6.5
Small Communications
System (Key
Telephone System)
Services
7.1
7.2
Installation Service
Engineering Service
7.3
7.4
Maintenance Service
Repair Service
Note 1
Note 2
Note 3
Examples:
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-19
Appendix A
Category:
Definition:
Examples:
Customer Support
Service
7.6
Procurement Services
7.7
Logistical Services
7.8
7.9
General Support
Service
Components
and
Subassemblies
8.1
Components
8.2
Subassemblies
8.2.1
Simple
Note 1
Note 2
Note 3
Typically includes
refurbishing/retesting
Typically includes strictly
warehousing and transportation
VCXOs
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-20
Appendix A
Category:
Definition:
Examples:
Medium Complexity
8.2.3
High Complexity
8.2.4
More than 50 components or 240 solder connections but less than 501
components or 2401 solder connections excluding connectors
More than 500 components or 2400 solder connections excluding
connectors
Note 1
Note 2
Note 3
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
Bolded text in the product category definition indicates the primary function of the product category. This is the function to
use for outage measurements.
TL 9000 Quality Management System Measurements Handbook 3.0
A-21
Appendix A
3.
3.1
3.2
Where the normalization factor is traffic capacity based, such as DS1, OC-1, DSL
or Terminations, the calculation shall be based on the true useable traffic
capacity. Equipment within the system used to provide protection for the main
traffic path shall not be included, as it does not add useable capacity to the
system.
Software measurements are based on the three most dominant releases.
% = 100 x Quantity Defective / Total Quantity. % is applicable to "Software
Only" measurements.
NA means the measurement is not applicable for the product category.
None means that no common normalization factor has been identified for the
product category; however, data shall be submitted for the measurement.
The column headings in Table A-2 are general descriptions covering several submeasurements in some cases. For cross-references to the detailed descriptions
of the measurements elsewhere in this document, find measurement/ submeasurement symbols in Table A-5 Measurement Summary Listing.
3.3
Table A-5 is a listing of the measurements included in this handbook with the
symbols used in data reporting, the applicability to hardware, software, and/or
services (H, S, V), and a reference to the table in this handbook with data
reporting details. The symbols listed here are referenced by the normalization
unit and applicability table to clarify the general descriptions used as column
headings
Appendix A
Product Category
Code
Description
1.2
1.2.1
Packet Switch
Public Packet Switched Network
(PPSN)
1.2.2
IP Packet Switch/Router
1.2.3
1.2.4
2
2.1
Signaling
Service Control Point (SCP)
Note 1
Note 2
Note 3
Note 4
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
Minutes/
system/
year
Outages/
system/
year
Minutes/
system/
year
Minutes/
NC/
year
Minutes/
system/
year
Minutes/
system/
year
Outages/
system/
year
Outages/NC/
year
Minutes/
system/
year
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Returns/
termination/
year
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
Outages/
system/
year
Outages/
system/
year
Returns/
termination/
year
Returns/
termination/
year
Returns/
system/
year
Returns/
termination/
year
Outages/
system/
year
Returns/
system/
year
Problem
Reports/
system/year
Return Rate
H
RR (all)
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-23
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
Minutes/
system/
year
NA
Outages/
system/
year
NA
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Returns/
system/
year
NA
Problem
Reports/
system/year
Problem
Reports/
system/year
Return Rate
H
RR (all)
2.3
Transmission
3.1
3.1.1
3.1.1.1
3.1.1.1.1
Outside Plant
Transmission Medium
Metallic Cable Products
Metallic Conductor Cable
NA
NA
NA
None
NA
NA
NA
NA
3.1.1.1.2
Metallic Connectors
NA
NA
NA
Problem
Reports/
unit shipped/
year
NA
NA
NA
NA
3.1.1.2
3.1.1.2.1
NA
NA
NA
None
NA
NA
NA
NA
Note 1
Note 2
Note 3
Note 4
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-24
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
NA
NA
3.1.1.3
3.1.1.3.1
Transmission Sub-systems
Active Sub-systems
NA
NA
3.1.1.3.2
NA
NA
3.1.1.3.3
Ancillary Sub-systems
NA
NA
3.1.2
3.1.2.1
Physical Structure
Enclosures
NA
NA
Note 1
Note 2
Note 3
Note 4
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
NA
Problem
Reports/
unit shipped/
year
NA
NA
NA
NA
Returns/
unit/
year
Returns/
unit/
year
Returns/
unit/
year
Problem
Reports/
unit/year
Problem
Reports/
unit/year
Problem
Reports/
unit/year
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
NA
Returns/
unit/
year
Problem
Reports/
unit shipped/
year
NA
NA
NA
NA
Return Rate
H
RR (all)
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-25
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
NA
NA
Returns/
unit/
year
Return Rate
H
RR (all)
3.1.2.3
Conduits
NA
NA
Returns/
unit/
year
3.2
3.2.1
3.2.1.1
Transport Equipment
Cross Connect Systems
Manual Cross Connect Systems
NA
NA
Returns/
DS1/ year
3.2.1.2
3.2.2
3.2.2.1
3.2.2.1.1
Carrier Systems/Multiplexers
Interoffice/Long Haul
Metallic Carrier System
Note 1
Note 2
Note 3
Note 4
Minutes/DS1/ Outages/DS1
year
/
year
Returns/
DS1/ year
Minutes/DS1/
year
Returns/
DS1/
year
Outages/
DS1/
year
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Problem
Reports/
unit shipped/
year
Problem
Reports/
meter shipped/
year
NA
NA
NA
NA
NA
NA
NA
NA
Problem
Reports/
system/year
Problem
Reports/
system/year
NA
NA
NA
NA
Problem
Reports/
system/year
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-26
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
Minutes/
OC-1/
year
Outage/
OC-1/
year
Returns/
OC-1/
year
Return Rate
H
RR (all)
3.2.2.1.2.2
WDM/DWDM/Optical Amplification
Products
Minutes/
OC-1/
year
Outages/
OC-1/
year
Returns/
OC-1/
year
3.2.2.1.3
Microwave
Minutes/
DS1/
year
Outages/
DS1/
year
Returns/
DS1/
year
3.2.2.2
Loop Carrier
3.2.3
Line Terminating
Equipment/Distributing Frames
Minutes/
DS1/
year
NA
Outages/
DS1/
year
NA
Returns/
DS1/
year
Returns/
termination /
year
Note 1
Note 2
Note 3
Note 4
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Problem
Reports/
network
element/year
Problem
Reports/
network
element/year
Problem
Reports/
network
element/year
Problem
Reports/
DS1/year
Problem
Reports/
termination/
year
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-27
Appendix A
Product Category
Code
Description
3.3
3.3.1
Wireless Transmission
Base Station Controller (BSC) and
Base Station System (BSS)
3.3.2
3.3.3
4
4.1.
4.1.1
Note 1
Note 2
Note 3
Note 4
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
NA
NA
Minutes/
system/
year
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Returns/
DSL line/
year
Problem
Reports/
DSL line/year
Outages/
system/
year
Returns/
unit/
year
Problem
Reports/
system/year
Minutes/
system/
year
Minutes/
system/
year
Outages/
system/
year
Outages/
system/
year
Returns/
unit/
year
Returns/
unit/
year
Problem
Reports/
system/year
Problem
Reports/
system/year
NA
NA
Returns/
unit/
year
Problem
Reports/
system/year
Return Rate
H
RR (all)
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-28
Appendix A
Product Category
Code
Description
4.1.3
4.2
4.2.1
4.2.2
On Line Non-Critical
4.2.3
Off Line
4.3
Note 1
Note 2
Note 3
Note 4
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
NA
NA
Minutes/
system/
year
Outages/
system/
year
Minutes/
system/
year
Minutes/
system/
year
Minutes/
system/
year
NA
Outages/
system/
year
Outages/
system/
year
Outages/
system/
year
NA
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Returns/
unit/
year
NA
Problem
Reports/
system/year
Problem
Reports/
system/year
Returns/
system/
year
Returns/
system/
year
Returns/
system/
year
NA
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
None
NA
NA
NA
NA
Return Rate
H
RR (all)
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-29
Appendix A
Product Category
Code
Description
5.2
5.3
Power Systems
6
6.1
6.1.1
Customer Premises
Enhanced Services Platforms
Interactive Voice Response (IVR)
Platforms
6.1.2
Messaging Platforms
6.1.3
Multi-Application Platforms
6.2
Terminal Equipment
Note 1
Note 2
Note 3
Note 4
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
Minutes/
system/
year
Minutes/
system/
year
Minutes/
system/
year
Outages/
system/
year
Outages/
system/
year
Outages/
system/
year
Minutes/
system/
year
Minutes/
system/
year
Minutes/
system/
year
Outages/
system/
year
Outages/
system/
year
Outages/
system/
year
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Returns/
system/
year
Returns/
system/
year
Returns/
unit/
year
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
NA
NA
NA
NA
NA
NA
NA
NA
Returns/
system/
year
Returns/
system/
year
Returns/
system/
year
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
Return Rate
H
RR (all)
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-30
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
NA
NA
Returns/
unit/
year
Return Rate
H
RR (all)
6.2.1.2
NA
NA
Returns/
unit/
year
6.2.2
Fax Equipment
NA
NA
Returns/
unit/
year
6.2.3
Data Modems
NA
NA
Returns/
unit/
year
6.2.4
NA
NA
Returns/
unit/
year
Note 1
Note 2
Note 3
Note 4
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Problem
Reports/
unit shipped/
year
Problem
Reports/
unit shipped/
year
Problem
Reports/
unit shipped/
year
Problem
Reports/
unit shipped/
year
Problem
Reports/
unit shipped/
year
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-31
Appendix A
Product Category
Code
Description
6.5
Note 1
Note 2
Note 3
Note 4
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c
r,h,OFMs,c
Minutes/
system/
year
Minutes/
system/
year
Minutes/
system/
year
Outages/
system/
year
Outages/
system/
year
Outages/
system/
year
Return Rate
H
RR (all)
Returns/
system/
year
Returns/
system/
year
Returns/
system/
year
Software Only
(Per Applicable Option)
Problem
Reports
H,S
NPR (all)
Corrective
Patch
Quality
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
IPR (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Problem
Reports/
system/year
Problem
Reports/
system/year
Problem
Reports/
system/year
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-32
Appendix A
Table A-2
Product Category
Code
Description
TL 9000 Measurement Symbols (see Table A-5)
RQMS Alternative Symbols (see Table A-5)
7
Services
7.1
Installation Service
7.2
Engineering Service
7.3
Maintenance Service
7.4
Repair Service
7.5
7.6
Procurement Services
7.7
Logistical Services
7.8
7.9
Table A-2
Note 1
Note 2
Note 3
Note 4
Service Quality
SQ
NA
Return Rate
RR (all)
Problem Reports/
job/year
Problem Reports/
job/year
Problem Reports/
unit maintained/
year
Problem Reports/
unit repaired/
year
Problem Reports/
1000 requests/
year
Problem Reports/
unit/year
Problem Reports/
unit/year
% audits conforming
NA
NA
NA
NA
% of successful repairs
NA
NA
NA
NA
Returns/
unit/year
NA
NA
Problem Reports/
unit /year
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-33
Appendix A
Product Category
Code
Description
Downtime
Performance
H, S
SO2; SO4;
Outage
Frequency
H, S
SO1;SO3;
r,h,DPMs,c_
r,h,OFMs,c
Return
Rate
H
RR (all)
Software Only
CPQ (all)
Feature
Patch
Quality
FPQ (all)
Software
Update
Quality
SWU (all)
Release
Application
Aborts
RAA (all)
DPQ (all)
DFP (all)
DSU (all)
RAQ (all)
Problem
Reports
H,S
NPR (all)
Corrective
Patch Quality
IPR (all)
8.1
Components
NA
NA
NA
Problem
Reports/
unit shipped/
year
NA
NA
NA
NA
8.2
8.2.1
Subassemblies
Simple
NA
NA
Returns/
unit/
year
NA
NA
NA
NA
8.2.2
Medium Complexity
NA
NA
Returns/
unit/
year
NA
NA
NA
NA
8.2.3
High Complexity
NA
NA
Returns/
unit/
year
NA
NA
NA
NA
8.2.4
NA
NA
Returns/
unit/
year
Problem
Reports/
unit shipped /
year
Problem
Reports/
unit shipped/
year
Problem
Reports/
unit shipped /
year
Problem
Reports/
unit shipped /
year
NA
NA
NA
NA
Note 1
Note 2
Note 3
Note 4
The information in this table may have changed. See the QuEST Forum web site, http://www.questforum.org/ for the latest
information.
Measurements FRT, OFR & OTD are applicable and must be reported for all product categories except for OTD for 7.5.
Product Categories listed in RED and italicized will be used for possible Data Aggregation only. Measurements must be
submitted per the lower Product Category listing.
If the normalization factor contains the word shipped, then the quantity shipped in the 12 months ending prior to the
month being reported shall be used.
TL 9000 Quality Management System Measurements Handbook 3.0
A-34
Appendix A
4.
Equivalency Tables
Frequency
Equivalent
NORTH AMERICAN
64 Kb
1 Trunk
1.544 Mb
24 DS0
1.728 Mb
1 DS1, 24 DS0
3.152 Mb
2 DS1, 48 DS0
6.312 Mb
4 DS1, 96 DS0
44.736 Mb
28 DS1, 672 DS0
51.84 Mb
1 DS3, 28 DS1, 672 DS0
155.52 Mb
3 DS3, 84 DS1, 2,016 DS0
622.08 Mb
12 DS3, 336 DS1, 8,064 DS0
2488.32 Mb
48 DS3, 1,344 DS1, 32,256 DS0
9953.28 Mb
192 DS3, 5,376 DS1, 129,024 DS0
INTERNATIONAL (PDH)
2,048 Mb
30 64 Kb Channels
8,448 Mb
4 2 Mbit/s, 120 64 Kb Channels
34,368 Mb
4 8 Mbit/s, 16 2 Mbit/s, 480 64 Kb Channels
139,264 Mb 4 34 Mbits/s, 64 2 Mbit/s, 1,920 64 Kb Channels
636,000 Mb
4 140 Mbit/s, 16 34 Mbit/s, 64 8 Mbit/s, 256 2
Mbit/s, 7,680 64 Kb Channels
Appendix A
Electrical
OC-1
OC-3
OC-12
OC-48
OC-192
STS-1
STS-3
STS-12
STS-48
STS-192
OC-768
OC-1536
Not available
STM-1o
(OC-3)
STM-4o
(OC-12)
STM-16o
(OC-48)
STM-64o
(OC-192)
Not applicable
Not applicable
Not applicable
Not applicable
Not applicable
STM-1e
STM-4e
Frequency
Equivalent
NORTH AMERICAN (SONET)
51.84 Mb
1 OC-1, 1 DS3, 28 DS1, 672 DS0
155.52 Mb
3 OC-1, 3 DS3, 84 DS1, 2,016 DS0
622.08 Mb
12 OC-1, 12 DS3, 336 DS1, 8,064 DS0
2,488.32 Mb 48 OC-1, 48 DS3, 1,344 DS1, 32,256 DS0
9,953.28 Mb 192 OC-1,192 DS3, 5,376 DS1, 129,024
DS0
39,680 Mb
Not available
158,720Mb
Not available
INTERNATIONAL (SDH)
155.52 Mb
1 E4, 4 E3, 64 E1, 1,920 Channels
622.08 Mb
STM-16e
2,488.32 Mb
STM-64e
9,953.28 Mb
VC-11 (VT1.5)
VC-12 (E1)
VC-2 (VT6)
VC-3 (E3)
VC-4 (E4)
1 E3 (34 Mb)
1 E4 (140 Mb)
Appendix A
5.
Table A-5 is a listing of the measurements included in this handbook showing: (1)
the symbols used in data reporting, (2) the applicability to hardware, software,
and/or services (H, S, V), and (3) a reference to the table with data reporting
details. The symbols listed here are also included in Table A-2, Measurement
Applicability Table (Normalized Units), to clarify the general descriptions in the
column headings.
5.1
5.2
5.3
H,S
compared
NPR2
H,S
compared
NPR3
H,S
compared
NPR4
compared
IPR
H,S
IPR1
H,S
compared
IPR2
H,S
compared
IPR3
H,S
compared
FRT
H,S,V
5.2-3
FRT2
H,S
compared
FRT3
H,S
compared
FRT4
compared
ORT
H,S
ORT2
ORT3
OFR
5.1-4
H,S
H,S
H,S,V
5.2-4
compared
compared
5.3-3
Appendix A
5.4
6.1
H,S
research
OFR4
research
OPR
H,S
5.3-4
OPR2
H,S
research
OPR3
H,S
research
OTD
H,S,V
OTIS
OTI
OTS
SO
H,S,V
H,S
V
H,S
5.4-2
compared
compared
compared
6.1-4
SO1
SO2
SO3
H,S
H,S
H,S
compared
compared
compared
SO4
H,S
H,S
compared
SOE
6.1-5
rDPMs
H,S
compared
hDPMs
H,S
compared
rDPMc
H,S
compared
hDPMc
H,S
compared
rOFMs
H,S
compared
hOFMs
H,S
compared
rOFMc
H,S
compared
Appendix A
7.1
8.1.5
8.1.6
H,S
6.1-6
DPM
H,S
compared
DPMs
H,S
compared
OFM
OFMs
H,S
H,S
compared
compared
RR
H
IRR
YRR
LTR
NYR
RAA
RAA0
RAA1
RAA2
RAQ
H
H
H
H
S
S
S
S
S
7.1-2
research
research
research
compared
8.1.5-3
compared
compared
compared
8.1.5-4
RAQ0
compared
RAQ1
compared
RAQ2
compared
Rar0
compared
Rar1
compared
Rar2
compared
CPQ
S
CPQ0
8.1.6-3
compared
Appendix A
8.1.6
8.1.7
compared
8.1.6-4
DCP0
compared
DCP1
compared
DCP2
compared
CPr0
compared
CPr1
compared
CPr2
compared
FPQ
S
FPQ0
FPQ1
FPQ2
DFP
S
S
S
S
8.1.6-3
research
research
research
8.1.6-4
DFP0
research
DFP1
research
DFP2
research
FPr0
research
FPr1
research
FPr2
research
SWU
S
SWU0
SWU1
SWU2
DSU
S
S
S
S
8.1.7-3
compared
compared
compared
8.1.7-4
Appendix A
9.1
Appendix B
Appendix B
B. Measurements
Profile for
Customer
Satisfaction
Mechanism
1.1
Purpose
These measurements are used to measure and improve the degree of customer
satisfaction with an organization and its products from a customer point of view to
help the organization to improve the satisfaction of its customers.
1.2
All products delivered through a purchase order and fulfillment process are
applicable. This should include stock items as well as items that are made-toorder.
1.3
Detailed Description
1.4
Sources of Data
Both customers and supplier organizations should administer the mechanism for
determining customer satisfaction. Results should be obtained at least once per
Appendix B
year and reported according to each customer or organization firms own formats
and procedures.
1.6
Example
Delivers on time
Quality of Pricing
Competitive pricing
Price stability
Price accuracy
Market insight
Invoicing efficiency
Order acknowledgement
Quality of Product
Glossary
Glossary
A&M
ABS
ACD
ACD
Afactor
AIN
AOJD
ATM
BSC
BSS
BTS
CCS
CO
COT
CPQ
CRCD
CRD
CROJD
DCS
DIS
DPM
DS(x)
DSX
DWDM
E(x)
FAX
FDF
FDI
FPQ
FRT
FRU
GA
H
H/S
HLR
IRR
Glossary
IP
IP
IR
ISDN
IVR
LEC
LOR
LSTP
LTR
MD
MSC
MTBF
NA
NGDLC
NPR
NTF
NYR
OC-(xxx)
OFM
ONU
OPR
OSS
OTD
OTI
OTIS
OTS
PBX
PDH
PO
POTS
RAA
RQMS
RSTP
RT
S
SCP
SDH
SFAR
SLC
SO
SONET
SPC
SQ
SS7
Glossary
SSP
STM-(x)e
STM-(x)o
STP
STS
SWIM
SWU
TCAP
UDLC
V
VC
VT
WDM
xDLC
xDSL
YRR
Glossary
Note: The following terms are used in this handbook or in the companion TL 9000 Quality Management
System Requirements Handbook.
Accredited Registrars
Annualization Factor
(Afactor)
Afactor
12
13
10.4
8.7
13.04
12.59
12.17
11.77
Certification
Certification Mark
Closure Criteria
Closure Date
Closure Interval
Compared Data
Glossary
Configuration
Management
Customer Base
Design Change
Deviation
Disaster Recovery
Engineering Complaint
Electrostatic Discharge
Field Replaceable
Unit
Fix
Fix Response
Time
The interval from the receipt of the original problem report to the
organizations first delivery of the official fix.
General Availability
Installation and/or
Engineering Audit
Installed System
Glossary
Maintenance
Manufacturing
Discontinued
A product at the end of its life cycle that is no longer generally available.
Measurement
Method
No Trouble Found
Normalization Factor
Official Fix
On-Time Installed
System Delivery
Patch
Glossary
Patch Defective
Corrective
A patch that:
a) cannot be installed,
b) fails to provide the intended feature capability,
c) is withdrawn because of potential or actual problems,
d) causes an additional critical or major problem.
Patch - Official
Plan
Problem Escalation
Glossary
Problem Report
Product Category
Program
QuEST Forum
Registrar
Release Application
Reliability
Research Data
Return
Risk Management
Glossary
Scheduled Outage
Service Categories
Service Providers
Severity Level
Subscriber
Support Service
Transaction
The complete cycle from a service request through the completion of the
service by the supplier.
System Test
Temporary Fix
A fix that is delivered to a limited number of systems in the field for the
purposes of verification or to solve system problems requiring immediate
attention. A temporary fix is usually followed by an official fix.
Test Plan
Virus, Software
Work Instructions
Glossary
N
nonconformity 3.6.2
O
objective evidence 3.8.1
organization 3.3.1
organizational structure 3.3.2
P
preventive action 3.6.4
procedure 3.4.5
process 3.4.1
product 3.4.2
project 3.4.3
Q
qualification process 3.8.6
qualified auditor 3.9.14
quality 3.1.1
quality assurance 3.2.11
quality characteristic 3.5.2
quality control 3.2.10
quality improvement 3.2.12
quality management 3.2.8
quality management system 3.2.3
quality manual 3.7.4
quality objective 3.2.5
quality plan 3.7.5
quality planning 3.2.9
quality policy 3.2.4
R
record 3.7.6
regrade 3.6.8
release 3.6.13
repair 3.6.9
requirement 3.1.2
review 3.8.7
rework 3.6.7
information 3.7.1
infrastructure 3.3.3
inspection 3.8.2
interested party 3.3.7
scrap 3.6.10
specification 3.7.3
supplier 3.3.6
system 3.2.1
M
management 3.2.6
management system 3.2.2
measurement control system 3.10.1
measurement process 3.10.2
measuring equipment 3.10.4
metrological characteristic 3.10.5
metrological confirmation 3.10.3
metrological function 3.10.6
T
technical expert <audit> 3.9.12
test 3.8.3
top management 3.2.7
traceability 3.5.4
V
validation 3.8.5
verification 3.8.4
W
work environment 3.3.4
Bibliography
Bibliography
1. GR-929-CORE
2. GR-230-CORE
3. GR-1323-CORE
4. ISO 8402:1994
5. TR-NWT-000284
6. TL 9000
7. ISO 9000:2000