Tier Classifications Define Site

Infrastructure Performance
By W. Pitt Turner IV, P.E., John H. Seader, P.E., and Kenneth G. Brill

Widely accepted within the uninterruptible industry, The Uptime Institutes Tier Performance Standards
are an objective basis for comparing the capabilities of a particular design topology against others or to
compare groups of sites. This paper defines a four Tier system providing discussion and illustrations of
each classification. Significant cautions about Tier misapplication are provided. While the paper focuses
primarily on design topology, sustainability (how the site is operated once constructed) plays a more
significant role in what site availability is actually achieved. Actual site performance figures combining
both design topology and sustainability are presented by Tier classification.

hardware reliability. Information technology customers

This white paper:
expect availability of Five Nines or 99.999%.
n Equips non-technical managers with a simple and Unfortunately, the substantial investment a business
effective means for identifying different data center frequently makes to achieve Five Nines in its computer
site infrastructure design topologies. hardware and software platforms is likely to be
n Provides IT based definitions and performance insufficient unless matched with a complementary site
requirements for each Tier Level. infrastructure that can support their availability goals.
n Provides actual 5-year availability for 16 major The site infrastructure includes 16 power, cooling, and
sites by Tier classification. other critical physical layer environmental sub-systems
n Warns that site availability is a combination of that must work together as a tightly integrated uptime
both design topology and sustainability with system.
considerable optimization art involved.
n Warns that component/system counts or MTBF Tier History
analysis plays no role in determining Tier The Uptime Institute, Inc. (Institute) developed a
compliance partially because each fails to include four tiered classification approach to site infrastructure
sustainability factors which account for 70% of all functionality that addresses the need for a common
infrastructure failures. benchmarking standard. The Institutes system has been
n Cautions self proclaimed Tier claims all too often in use since 1995 and has become the default standard for
turn out to be misleading, incomplete, or wrong. the uninterruptible uptime industry. An early-1990s Tier
n Outlines need for third-party validation of site predecessor outlined seven ways of distributing critical
selection, design, and sustainability decisions power to the computer equipment, but was not simple
before committing to multi-million dollar projects. and all inclusive. A broader standard was required.
n Provides a commentary on typical Tier attributes.
Creation of the Institutes original Tier definition
was stimulated by multiple industry requests. Senior
Background management decision makers needed a simple and
One of the most common sources of confusion in the field effective non-technical means of conveying the
of uninterruptible uptime is what constitutes a reliable differences in data center investments. Since the
data center. All too often, reliability is in the eye of the original pioneering work done more than 10 years
beholderwhat is acceptable to one person or company ago, the Tier concept has been further developed and
is inadequate to the next. Competing companies with data validated by broad industry use. The Institutes objective
centers of radically different infrastructure capabilities performance-based standard is very useful in ensuring
are all claiming to deliver high availability. a consistent framework to compare various alternatives
companies may consider for obtaining data center space.
With the continuously increasing pressure on high These include such options as owned, leased, third party
availability comes an increased demand for computer providers, and so on.

1
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Site Availability As Actually Experienced The advent of dual-powered computer hardware

By Information Technology in tandem with Tier IV electrical and mechanical
The following tier commentary includes actual measured infrastructure is an example of site infrastructure design
results for site availability ranging from 99.67% to more and computer equipment design working together to
than 99.99%. achieve higher availability. Even with the significant
improvements in computer hardware design made over
These figures are not predictive of future site results, the past 10 years, many data centers constructed in the
but do reflect actual operating experience at a specific last 5 years, and even today, claim Tier IV functionality,
list of sites representing the four Tiers of functionality. but actually deliver only Tier I, II, or III. This constrains
It is important to note that this range of availability their capability to match the availability required by
is substantially less than the current Information the information technology they support. The purpose
Technology (IT) expectations of Five Nines. This leads of this paper is to outline what it takes to consistently
to the conclusion that site availability limits overall IT meet the requirements of the different tier levels.
availability.
The Need for Third-Party Certification Is a
Four Tier Levels Reflect Evolution of Data Growing Self-Preservation Requirement
Center Uptime Objectives In site infrastructure design and operation, the devil
Over the last 40 years, data center infrastructure designs is in the details and the truth about a particular design
have evolved through at least four distinct stages, topology will ultimately come out, but all too often after
which are captured in the Institutes classification the warranty period has expired. When this happens,
system. Historically, Tier I first appeared in the early it can be a career ending event. Forensic investigation
1960s, Tier II in the 1970s, Tier III in the late 1980s by the Institute into thousands of Abnormal Incidents
and early 90s, and Tier IV in 1994. The Institute over the last 12 years indicates that at least five and
participated in the development of Tier III concepts and often seven interacting problems are required before a
pioneered in the creation of Tier IV. Tier IV electrical downtime failure occurs. The database upon which this
power distribution systems were made possible, in analysis is built is in unique in the world.
part, by Ken Brill, Executive Director of the Institute.
In 1991, he envisioned a future when all computer Increasingly, senior executives desire to have their
hardware would come with dual power inputs. This critical sites independently certified as being compliant
became US Patent 6,150,736. United Parcel Services to the Tier standards. This provides a validation that
1994 Windward data center project was the first Tier IV the technical details of what the designer designed and
design. During construction of the Windward project, the contractor built is actually what the owner wanted.
United Parcel Service worked with IBM and other When project designers self proclaim a site meets a
computer hardware manufacturers to provide dual- certain tier level or capacity, it is all too often inaccurate
powered computer hardware1. or only partly factual. The results can often be tragic
involving unnecessary downtime and tens of millions in
Tier IV technology requires having at least two completely unforeseen upgrade expense.
independent electrical systems. These dual systems supply
power through diverse power paths to the computer Certification is a service performed by The Uptime
equipment. This effectively moves the last point of electrical Institute, who is uniquely qualified to interpret and
redundancy from the Uninterruptible Power Supply apply the standards since the Institute created the
(UPS) system downstream to a point inside the computer underlying technology concepts that allowed the
hardware itself. Brills intuitive conclusion has since been standards to develop in the first place. In addition, the
confirmed by Institute research that has determined that Institute also brings awareness of emerging downtime
of the vast majority of site infrastructure electrical failures problems and trends at least three to five years before
occur between the UPS and the computer equipment. Since they are commonly recognized and addressed by the
completion of the Windward project in 1994, System plus rest of the industry.
SystemSM (S+S) Tier IV electrical designs have become
common and the number of computer devices with dual Site Certification by The Uptime Institute involves two
inputs has grown dramatically. There are exact parallels in separate, interrelated activities. The first is verification
the mechanical systems design. of the design topology and how it complies with the

1
There are 13 technical requirements to describe what is commonly called dual power. The actual details and additional dual power information can be found in the
Institutes white paper Fault-Tolerant Power Certification Is Essential When Buying Products for High-Availability which may be found at www.upsite.com/whitepapers.

2
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Tier standards. The second phase is verification of site If one unit of capacity is required to support
sustainability. While a particular topology design may the computer equipment, more than one unit of
meet the literal requirement of a Tier level, the lifecycle capacity is installed. Terms such as N+1 or N+2 are
effectiveness of that design may be extremely limiting commonly applied.
typically less than five years. Sustainability includes n Useable capacity: This is the maximum amount of
site selection; lifecycle effectiveness of the design load that can be applied to the N level of capacity.
topology and its transparent flexibility/scalability; ease Typically, the maximum amount of useable load is
of use; staffing level and coverage, training, and skills less than the non-redundant capacity to allow for
development; management procedures and processes; component aging, installation errors, and to provide
metrics and dashboards; commissioning and maintenance a contingency for unexpected demands.
practices; and the integration of the site infrastructure with n Useable capacity: This is the maximum amount of
the IT architecture. Human factors are important because load that can be applied to the N level of capacity.
70% or more of all site failures involve people. Of these Typically, the maximum amount of useable load is
failures, 2/3 are management error and 1/3 is human error. less than the non-redundant capacity to allow for
Human sustainability factors will largely determine the component aging, installation errors, and to provide
actual level of site availability achieved. a contingency for unexpected demands.
n Site infrastructure: This comprises all of the site
Previous Tier Level Information Is Now facility that includes the central plant plus the
Divided into TIER PERFORMANCE equipment that supports the power and cooling in
STANDARDS and COMMENTARY the computer room. It is important to remember that
Sections a typical data center site is composed of at least 20
Responding to user questions and concerns, this white major mechanical, electrical, fire protection, security
paper has been updated where appropriate and reorganized and other systems. Each has additional subsystems
into two separate sections: and components.
n The TIER PERFORMANCE STANDARDS are now in n Fault tolerant: This means that a system can sustain
a totally separate section, similar to many engineering a worst case, unplanned event and not disrupt the
documents. The standards focus on the definitions of end user. The fault tolerant concept originated
the Tiers and the performance confirmation tests for in the IT environment. In the site infrastructure
determining compliance to the definitions. These are world, it means that the computer equipment will
absolute criteria. Performance is measured by outcome not be impacted by a facility failure. This requires
confirmation tests and operational results. This is totally multiple sources and multiple distribution paths so
different than a prescriptive approach or a specific list of a failure on one source or path does not impact the
equipment not guaranteeing a performance outcome. other. This also requires use of computer equipment
n The TIER COMMENTARY focuses on examples of that meets the Institutes Fault Tolerant Compliant
the various ways to design and configure each Tier. In Power Specification. Computer equipment that
addition, the commentary section includes discussion and does not meet that specification requires additional
examples to aid in Tier understanding and information components, such as a point-of-use switch. During
on common design topology failures. A comparison site infrastructure maintenance activity, the risk of
table of typical Tier attributes, availability and cost are disruption may be elevated.
provided. The commentary section also offers guidance n Concurrent maintainability: Originally, this was also
in the comprehension, design, implementation, and the an IT term. It means any work can be performed on
use of the Tier definitions. a planned basis without impacting the end user. In
the site infrastructure world, this means that ANY
Definition of Terms Used in the TIER capacity component or distribution element can be
PERFORMANCE STANDARDS and TIER repaired, replaced, serviced, tested, etc., without
COMMENTARY Sections impacting the computer equipment.
n Computer equipment: This is a broad phrase encompassing
all information technology equipment required at a data TIER PERFORMANCE STANDARD
center to perform the information processing work.
It includes servers, storage, network, and all other Tier I: Basic Site Infrastructure
information technology components. The fundamental requirement
n Redundant capacity components: The components n A Tier I basic data center has non-redundant capacity
beyond the number of capacity units required to support components and single non-redundant path distribution
the computer equipment are referred to as redundant. paths serving the sites computer equipment.

3
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

The performance confirmation test(s) serving the sites computer equipment. Generally, only
n Any capacity component or distribution path failure one distribution path serves the computer equipment at
will impact the computer systems. any time.
n Planned work will require most or all of the systems
to be shut down, impacting the computer systems. The performance confirmation test
n Each and every capacity component and element of
The operational impact the distribution paths can be removed from service on
n The site is susceptible to disruption from both planned a planned basis without causing any of the computer
and unplanned activities. equipment to be shut down.
n The site infrastructure must be completely shut down on
an annual basis to safely perform necessary preventive The operational impact
maintenance and repair work. Urgent situations may n The site is susceptible to disruption from unplanned
require more frequent shutdowns. Failure to perform activities.
this maintenance work increases the risk of unplanned n Planned site infrastructure maintenance can be
disruption as well as the severity of the consequential performed by using the redundant capacity components
failure. and distribution paths to safely work on the remaining
n Operation errors or spontaneous failures of site equipment.
infrastructure components will cause a data center n In order to establish concurrent maintainability of the
disruption. critical power distribution system between the UPS
and the computer equipment, Tier III sites require all
Tier II: Redundant Capacity Components computer hardware have dual power inputs as defined
Site Infrastructure by the Institutes Fault Tolerant Power Compliance
The fundamental requirement Specifications Version 2. This document can be found
n A Tier II data center has redundant capacity components at http://www.upsite.com/TUIpages/tuifault_spec_2-
and single non-redundant distribution paths serving the 0.html. Devices such as point-of-use switches must
sites computer equipment. be incorporated for computer equipment that does not
meet this specification.
The performance confirmation test(s) n During maintenance activities, the risk of disruption
n A capacity component failure may impact the computer may be elevated.
equipment. n Operation errors or spontaneous failures of site
n A distribution path failure will cause the computer infrastructure components may cause a data center
equipment to shut down. disruption.

The operational impact Tier IV: Fault Tolerant Site Infrastructure

n The site is susceptible to disruption from both planned The fundamental requirement
activities and unplanned events. n A fault tolerant data center has redundant capacity
n Redundant UPS modules and engine generators are systems and multiple distribution paths simultaneously
required. serving the sites computer equipment.
n The site infrastructure must be completely shut down n All IT equipment is dual powered and installed
on an annual basis to safely perform preventive properly to be compatible with the topology of the sites
maintenance and repair work. Urgent situations architecture.
may require more frequent shutdowns. Failure to
perform this maintenance work increases the risk The performance confirmation test(s)
of unplanned disruption as well as the severity of n A single worst-case failure of any capacity system,
the consequential failure. capacity component or distribution element will not
n Operation errors or spontaneous failures of site impact the computer equipment.
infrastructure components may cause a data center n Each and every capacity component and element of
disruption. the distribution paths must be able to be removed from
service on a planned basis without causing any of the
Tier III: Concurrently Maintainable Site computers to be shut down.
Infrastructure n In order to establish fault tolerance and concurrent
The fundamental requirement maintainability of the critical power distribution system
n A concurrently maintainable data center has redundant
between the UPS and the computer equipment, Tier IV
capacity components and multiple distribution paths sites require all computer hardware have dual power

4
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

inputs as defined by the Institutes Fault Tolerant Power Simply put, the Tier rating for an entire site is limited
Compliance Specifications Version 2. This document to the rating of the weakest subsystem that will impact
can be found at http://www.upsite.com/TUIpages/ site operation. For example, a site with a robust Tier IV
tuifault_spec_2-0.html. Devices such as point-of-use UPS configuration combined with a Tier II chilled water
switches must be incorporated for computer equipment system will yield a Tier II site rating.
that does not meet this specification.
n Complementary systems and distribution paths must This is driven by the need to manage perception in senior
be physically separated (compartmentalized) to management, as well as to factually report actual site
prevent any single event from impacting both systems capabilities. If a site is advertised within an organization
or paths simultaneously. as being fault tolerant and concurrently maintainable
(Tier IV), it is intolerable to shut the site down at any
The operational impact time in the futureregardless of what subsystem may
n The site is not susceptible to disruption from a single have required the shut down.
unplanned worst-case event.
n The site in not susceptible to disruption from any There are no partial or fractional Tier ratings. The sites
planned work activities. Tier rating is not the average of the ratings for the 16
n The site infrastructure maintenance can be performed by critical site infrastructure subsystems. The sites tier rating
using the redundant capacity components and distribution is the LOWEST of the individual subsystem ratings.
paths to safely work on the remaining equipment.
n During maintenance activities, the risk of disruption Similarly, the Tier cannot be imputed by using
may be elevated. calculated Mean Time Between Failure (MTBF)
n Operation of the fire alarm, fire suppression, or the component statistical reliability to generate a predictive
emergency power off (EPO) feature may cause a data availability and then using that number to match
center disruption. the actual measured availability results shown later in
Figure 2. Even if statistically valid component values
Determining a Sites Tier Rating for existed (and they dont because product life cycles are
Design Topology getting shorter and shorter and no independent, industry-
Determining a sites actual Tier rating for design topology wide database exists to collect failures), this approach
is not a complicated process, although it is one that is fails to include people which consistently are involved in
rarely done correctly. Figure 1 graphically illustrates 70% of all site failures. A calculated reliability of 0.9999
the tier performance standards. For discussion of the which ignores human interaction does NOT define a site
standards, see the following commentary section. as being Tier IV. The only way to determine Tier Level is to

Figure 1:
Performance Standards by Tier Level

Tier Requirement Tier 1 Tier II Tier III Tier IV

Source System System System System + System

System Component N N+1 N+1 Minimum of N+1

Redundancy

Distribution Paths 1 1 1 normal and 2 simultaneously active

1 alternate
Compartmentalization No No No Yes

Concurrently Maintainable No No Yes Yes

Fault Tolerance No No No Yes

(single event)

5
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

objectively determine a sites ability to respond to planned infrastructure. Tier III and Tier IV site infrastructure
and unplanned events. solutions have an effective life beyond the current IT
requirement. Strategic site infrastructure solutions
TIER COMMENTARY enable the owner to make strategic business decisions
concerning growth and technology, unconstrained by
The Institutes STANDARDS Are current site infrastructure topology.
Outcome Based
The requirements used in the Institutes Tier Performance Tier III site infrastructure adds the concept of concurrent
Standard are necessarily and intentionally very broad to maintenance to Tier I and Tier II solutions. Concurrent
allow innovation in achieving the desired level of site maintenance means that any component necessary to
infrastructure performance, or uptime. The individual support the IT processing environment can be maintained
Tiers represent categories of site infrastructure topology without impact on the IT environment. The effect on the
that address increasingly sophisticated operating concepts, site infrastructure topology is that a redundant delivery
leading to increased site infrastructure availability. The path for power and cooling is added to the redundant
performance outcomes defining the four Tiers of site critical components of Tier II. Maintenance allows the
infrastructure are very straight forward. Recent initiatives equipment and distribution paths to be returned to like
by several groups to replace the Institutes Tier concepts new condition on a frequent and regular basis. Thus, the
with component counts and checklists has lost focus that system will reliably and predictably perform as originally
ultimately counts is uptime performance. Most designs intended. Moreover, the ability to concurrently allow
that will pass a checklist approach will absolutely fail a site infrastructure maintenance and IT operation requires
performance requirements approach. What this means is that any and every system or component that supports IT
that there is still considerable art to the science of uptime operations must be able to be taken offline for scheduled
and how sub-systems are integrated (or not integrated). maintenance without impact on the IT environment.
This concept extends to important subsystems such as
Tier Functionality Progression control systems for the mechanical plant, start systems
Tier I solutions acknowledge the owner/operators desire for engine generators, EPO controls, power sources for
for dedicated site infrastructure to support IT systems. cooling equipment and pumps, and others.
Tier I infrastructure provides an improved environment
compared to an office setting and includes: a dedicated Tier IV site infrastructure builds on Tier III, adding
space for IT systems; a UPS to filter power spikes, sags the concept of fault tolerance to the site infrastructure
and momentary outages; dedicated cooling equipment topology. Just like concurrent maintenance concepts,
that wont get shut down at the end of normal office fault tolerance extends to any and every system or
hours; and an engine generator to protect IT functions component that supports IT operations. Tier IV considers
from extended power outages. that any one of these systems or components may fail or
experience an unscheduled outage at any time. While
Tier II solutions include redundant critical power and the Tier IV definition is limited to consideration of a
cooling capacity components to provide an increased single system failure, Tier IV requires that the effect of
margin of safety against IT process disruptions from such a failure is considered on other site infrastructure
site infrastructure equipment failures. The redundant systems and components. For example, the loss of a
components are typically an extra UPS modules, cooling single switchboard will affect the operation of all the
units, chillers, pumps, and engine generators. Loss of equipment fed from that switchboard: UPS systems,
the capacity component may be due malfunction or to computer room cooling equipment, controls, etc.
normal maintenance.
The progressive nature of functionality from Tier I
Owners who select Tier I and Tier II solutions to through Tier II and Tier III to Tier IV is demonstrated
support current IT technology are typically seeking in the schematic illustrations found at the end of this
a solution to short-term requirements. Both Tier I and paper. The examples show the addition of components
Tier II are tactical solutions, usually driven by first-cost and distribution paths, as described above. Although
and time-to-market more so than life cycle cost and the illustrations shown are not recommended design
uptime (or availability) requirements. Rigorous uptime solutions for any particular set of requirements, the
requirements and long-term viability usually lead to four electrical topologies are illustrative of the Tier
the strategic solutions found in Tier III and Tier IV site classification concepts. Mechanical system functionally

6
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

progresses through the increasing Tiers similarly. Consistent, year) is not equivalent to a statistical reliability of 0.9995 (1
across-the-board application of Tier concepts for electrical, in 2,000 chance of a failure). Similarly, as outlined earlier, a
mechanical, automation and other subsystems is absolutely calculated statistical reliability of 0.9995 does not indicate a
required for any site to satisfy the Tier standards. site is better than Tier III.

Over the last few years, site infrastructure has been The Institute defines site availability from the
occasionally described by others in the industry in terms perspective of a user of IT. Any site incident or event
of fractional tiers (i.e. Tier 2.5), or incremental Tiers (Tier that affects information availability as experience by
III +, or Enhanced Tier III, or Tier IV light). Fractional end users detracts from site infrastructure availability.
or incremental descriptions for site infrastructure are The site downtime clock starts running from the
not appropriate. A site that has an extra UPS module, moment IT operations were first affected until they are
but needs all the installed computer room air handlers fully restored. Thus, site downtime is not the 15 seconds
running to keep the UPS room temperature within limits of a utility power failure, but the total time users were
does not meet site redundancy requirements for Tier II. A down until IT availability was restored. For Tier I and
switchboard that cannot be shutdown without affecting Tier II topologies, downtime for site infrastructure
more than the redundant number of secondary chilled maintenance (which includes the time to bring IT
water pumps is not concurrently maintainable (Tier III). systems down, perform the site maintenance, and
restore IT availability) typically has a bigger availability
IT Availability Success Is Dependent upon impact than a UPS system failure. Based on operating
Successful, Fully Integrated Operation of experience of monitored sites, the typical maintenance
All Site Infrastructure Systems outage at Tier I and Tier II sites is 12 hours. The time for
The Tier classifications were created to consistently IT to recover from a typical outage such as momentary
describe the site-level infrastructure required to sustain power loss is 4 hours at sites of any tier.
data center operations, not the characteristics of individual
systems or sub-systems. Data centers are dependent Tier I sites typically experience two separate 12-hour,
upon the successful operation of over 16 separate site site-wide shutdowns per year for maintenance or repair
infrastructure subsystems. Every subsystem and system work. In addition, on average, across multiple sites
must be consistently deployed with the same site uptime and over a number of years, Tier I sites experience 1.2
objective to satisfy the distinctive Tier requirements. equipment or distribution failures each year. The annual
The most critical perspective owners and designers must impact of maintenance and unplanned outages is 28.8
consider in making tradeoffs is what impact the decision hours per year, or 99.67% availability.
has on the integrated impact of the site infrastructure on
the IT environment in the computer room. Operations experience shows that, on average, Tier II
sites schedule three maintenance windows over a 2-year
The Institute has measured the actual availability, or period and have one unplanned outage each year. The
performance, of 16 data centers having site infrastructure redundant components of Tier II topology provide some
topologies meeting the four Tier definitions and has has maintenance opportunity leading to just one site-wide
established availability values representative of each shutdown each year, and reduce the number of equipment
classification. In practice, representative site availability, failures that affect the IT operations environment. The
stated as a percentage of annual operating time, is associated annual impact of maintenance and unplanned outages is
with each of the Institutes standard Tier classifications. 22 hours per year, or 99.75% availability.
These empirically determined values include sustainability
and human factors over a period of up to 10 years with uptime Tier III topology is concurrently maintainable, so annual
measured from the perspective of the IT clients operations maintenance shutdowns are not required, which allows
in the computer room. This real world site availability is an aggressive maintenance program improving overall
strikingly different than the probability of system failure equipment performance. Experience in actual data
that is often calculated using values from the Institute of centers show that operating better maintained systems
Electrical and Electronics Engineers (IEEE) Gold Book reduces unplanned failures to a 4-hour event every 2.5
for recommended practices for reliable power systems or years, or 1.6 hours on an annual basis. Tier III sites
guidelines from the IEEE Orange Book for emergency demonstrate 99.98% availability.
and standby power. A representative site infrastructure Tier IV provides robust, fault tolerant site infrastructure,
availability of 99.95% (about 4.4 hours of downtime per so that facility events affecting the raised floor are

7
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

empirically reduced to one 4-hour event in a 5-year level of performance. Unless human activity issues are
operating period, or 0.8 hours on an annual basis. Individual continually and rigorously addressed, at least one failure
equipment failures or distribution path interruptions may is likely over 5 years.
still occur, but the effects of the events are stopped short of
the IT operations environment. Tier IV sites consistently Typical Tier Attributes
demonstrate 99.99% availability. Tier I sites have their roots in the mainframe environments
of the 1970s. Tier IV became possible with the advent of
The representative availability percentages are a dual-powered computers in the 1990s. Tier II and Tier III
characteristic of the operating experience of multiple sites facilities were widespread in the 1980s; Tier III is the most
within each Tier classification. A site with a measured common site infrastructure currently being implemented
infrastructure availability of 99.90%midway between although most are designed for future transparent upgrade
Tier II (99.75%) and Tier III (99.98%)has an operating to Tier IV. Most owners find it fairly difficult to upgrade
experience consistent with sites having Tier II topology, by more than one tier level from what they previously had.
but does not achieve the availability of Tier III sites. A responsible approach to site infrastructure investment is
Availability does not determine the Tier classification. to understand clearly the availability objectives necessary
Even more importantly, infrastructure with a statistical to support the owners current and future business
probability of failure of 0.9990 cannot be represented as requirements, then to consistently design, build, and
a Tier 2.5 site, since the impact of the failure on overall operate the site to conform to those needs.
availability is not represented by the likelihood of a
system failure. The following chart (Figure 2) depicts various attributes
commonly associated with a particular Tier classification,
Independent of site infrastructure experience, IT but the attributes are not requirements of the Tier
organizations often describe data center availability definitions. For example, the presence of a raised floor
objectives as Five Nines, or 99.999% of uptime. This or any particular floor height are not criteria for any Tier.
is a very aggressive goal, especially if compared to the (The recommended height of raised floors, when used, is
observed consequences of a single site outage. While the most directly correlated to power density.)
site outage is assumed to be promptly restored (which
requires 24 by forever staffing), it can still require up Integration of IT Architecture and
to 4 hours for IT to recover information availability and Topology with Site Architecture and
restore end user functionality, even if the likelihood of a Topology Helps to Ensure Achieving
data base corruption or a server power supply failure are Uptime Objectives
set aside. In reality, facility failures often reveal previously There are many opportunities within the Information
unknown IT architecture, hardware, or software issues. Technology architecture to reduce or minimize the
impacts of these unfortunate site infrastructure failures.
If a momentary power outage results in a 4-hour end- These steps may include placing the redundant parts of
user disruption, how relevant is an objective of 99.999% the IT computing infrastructure in compartments served
availability? Based on a single site outage of 4 hours, it will by different site infrastructure systems so that a single
take 45.6 years of 100% uptime to restore cumulative site event cannot simultaneously affect all IT systems. Another
availability back to the 99.999% objective. (4 hours x 60 alternative is focusing special effort on business-critical
minutes an hour 5.26 minutes per year = 45.6 years.) and mission-critical applications so they do not require 4
hours to restore. These operational issues can improve the
Even a fault tolerant and concurrently maintainable Tier availability offered by any data center and are particularly
IV site will not satisfy an IT requirement of Five Nines important in a Four Nines data center housing IT
(99.999%) uptime. The best a Tier IV site hope for 100% equipment that requires Five Nines availability.
uptime for a string of multiple years. Figure 2 of Typical
Tier Attributes uses 99.995% for representative Tier IV The four Tier Standard classifications address topology,
site availability, but this assumes a site outage occurs not or configuration, of site infrastructure, rather than a
more than once every 5 years. With a properly designed prescriptive list of components, to achieve a desired
Tier IV configuration, the single event exposures that operational outcome. For example, the same number
can result in a site failure are the results of a fire alarm of chillers and UPS modules can be arranged on single
or the unintended operation of the EPO feature. Only power and cooling distribution paths resulting in a Tier II
the top 10 percent of Tier IV sites will achieve this (Redundant Components) solution, or on two distribution

8
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Figure 2:
Typical Tier Attributes

Tier 1 Tier II Tier III Tier IV

Building Type Tenant Tenant Stand-alone Stand-alone
Staffing None 1 Shift 1+Shifts 24 by Forever
Useable for Critical 100% N 100% N 90% N 90% N
Load
Initial Build-out Gross 20-30 40-50 40-60 50-80
Watts per Square Foot
(W/ft) (typical)
Ultimate Gross W/ft 20-30 40-50 100-1501,2,3 150+1,2
(typical)
Class A Uninterruptible No No Maybe Yes
Cooling
Support Space to 20% 30% 80-90+% 100+%
Raised Floor Ratio
Raised Floor Height 12 18 30-36 30-36
(typical)
Floor Loading lbs/ft 85 100 150 150+
(typical)
Utility Voltage (typical) 208, 480 208, 480 12-15 kV 12-15 kV
Single Points-of- Many + human Many + human Some + human None + fire and EPO
Failure error error error
Annual Site Caused IT 28.8 hours 22.0 hours 1.6 hours 0.8 hours
Downtime (actual field
data)
Representative Site 99.67% 99.75% 99.98% 99.99%
Availability
Typical Months to 3 3-6 15-20 15-20
Implement
Year first deployed 1965 1970 1985 1995
Construction Cost (+
30%)1,2,3,4,5
Raised Floor $220/ft $220/ft $220/ft $220/ft
Useable UPS Output $10,000/kW $11,000/kW $20,000/kW $22,000/kW

1 100 W/ft maximum for air-cooling over large areas, water or alternate cooling methods greater than 100 W/ft (added cost excluded).
2 Greater W/ft densities require greater support space (100% at 100 W/ft and up to 2 or more times at greater densities), higher raised
floor, and, if required over large areas, medium voltage service entrance.
3 Excludes land; unique architectural requirements, permits and other fees; interest; and abnormal civil costs. These can be several
million dollars. Assumes minimum of 15,000 ft of raised floor, architecturally plain, one-story building, with power backbone sized to
achieve ultimate capacity with installation of additional components or systems. Make adjustments for NYC, Chicago, and other high
cost areas.
4 Costs are based on 2005 data. Future year costs should be adjusted using ENR indexes.
5 See Institute White Paper entitled Dollars per kW plus Dollars per Square Foot Is a Better Data Center Cost Model than Dollars per
Square Foot Alone for additional information on this cost model.

9
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

paths that may result in a Tier III (Concurrently speed the recovery from site infrastructure incidents.
Maintainable) solution. Compare the Tier II and Tier III Less costly and less risky maintenance means the
diagrams at the end of this paper. Both topologies contain work is more likely to be completed, keeping the
the same N+1 capacity redundancy for engine generators equipment in better condition and calibration. More
and UPS modules, but the alternate distribution paths operations-centric designs make operations easier,
define the Tier III example. so fewer mistakes are made.

Applying the Standards Illustrative Examples

The Tier Performance Standard provides objective Some examples can illustrate site infrastructure
criteria to consistently evaluate the implementation of characteristics that impact sustainability, while not
the selected operational concepts in a design or existing affecting the overall Tier classification of the solution.
site infrastructure. The standard does not direct the
specific design solution or technology the owner or n A topology that can switch the power source
design team must use to reach the site performance for all mechanical components so they continue
objective. Owners are free to choose any number of running when any electrical panel is shut down
UPS configurations, products, or manufacturesas eliminates an operations constraint to maintenance.
long as the result can meet the target Tier classification. Procedures that require critical cooling equipment
Moreover, the use of static or rotary UPS systems, to be shut down during recurring electrical system
fuel cell technologies, direct expansion cooling, or maintenance may not be allowed if another chiller
air or water cooled chillers are left to the owner. The is out of service for repairs. Missed maintenance
Tier Standards have attained wide acceptance because leads to decreased reliability.
they allow the owner to include such concerns as first n A design that mounts critical components in difficult

cost, operations complexity, and product availability to reach areas or limits access space in the central
as appropriate, while still focusing on the desired plant may increase the time required to maintain
operational outcome of the completed facility. important systems. The increased time window may
eliminate the ability to schedule the maintenance
activity.
In addition to availability, other owner requirements
n Installing engine generators and switchgear inside
must be addressed in infrastructure design. Protection the facility (with adequate access space) eliminates
of data or physical assets is independent of the site the effects of weather and time-of-day on safe
infrastructure Tier classification. The increasing power maintenance and repair activities.
densities of IT equipment required other considerations n In order to improve stability, the combined load on a
than the redundancy in the power and cooling systems. critical system is often limited to 90% of non-redundant
Project elements like video surveillance and gaseous nameplate over a sustained period of time.
fire suppression are frequently necessary to meet n Compartmentalization, a Tier IV requirement,
an owners regulatory or insurance requirements, provides benefits for Tier III sites. The effects
completely separate from IT availability objectives. The of evacuation requirements for areas affected by
key understanding required for a successful data center refrigerant leaks can be limited to the number of
operation is to differentiate between Tier Performance redundant chillers by careful Compartmentalization.
Standard criteria, owner risk and cost tolerance, and Chillers that are necessary to keep the computer
Industry Best Practices. room cool can continue to operate while those in a
separate compartment are shut down to purge the
Consideration of cost, risk tolerance, and Best Practices refrigerant.
clearly point to a wider number of site infrastructure n Compartmentalization of the primary and maintenance
characteristics than Tier classification, alone. Experience electrical distribution paths also provides a major
with the Tier Standard since its inception indicates that advantage to a site. If an arc flash or electrical fire
Sustainability characteristics become an important (an unplanned event) occurred in a Tier III site, the
factor over time. Investments in Sustainability site could be disrupted. However, if the maintenance
characteristics account for much of the variance within path is physically separated from the normal path,
individual Tier solutions, often leading to increased compartmentalization would permit the site to
rapidly recover on a power path through a completely
availability. Typically, Sustainability characteristics
different space than where the fire occurred.
decrease the cost or risk of completing maintenance, or

10
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Each Industry Has a Unique Uptime

Need Driving the Site Infrastructure Tier Typical applications for Tier III facilities are
Level Required n Companies that support internal and external clients
After careful alignment of IT availability objectives with 24x7 such as service centers and help desks, but
site infrastructure performance expectations, an informed can schedule short periods when limited service is
company may select a site representing any of the Tier acceptable
classifications. Some considerations for selecting an n Businesses whose information technology resources
appropriate site infrastructure Tier are: support automated business processes, so client
impacts of system shutdowns is manageable
Tier I is appropriate for firms such as n Companies spanning multiple time zones with clients
n Small businesses where information technology and employees spanning regional areas
primarily enhances internal business process
n Companies who principal use of a web-presence is Companies selecting Tier III infrastructure usually have
as a passive marketing tool high-availability requirements for ongoing business, or
n Internet-based startup companies without quality of have identified a significant cost of disruption due to
service commitments a planned data center shutdown. These companies are
willing to accept the impact of disruption risk of an
These companies typically do not have an established unplanned event. However, Tier III is appropriate for
revenue stream or identifiable financial impact of companies who expect the functionality requirements to
disruption due to data center failure. Sometimes increase over time and do not want to abandon the data
companies with an established revenue steam will select center. Sometimes, these companies design a Tier III site
Tier I topology because their applications have a low to be upgraded to Tier IV.
availability requirement, such as only during a 5.5-
day business week. Other companies may select Tier Tier IV is justified most often for
I topology if they plan to abandon the site when the n Companies with an international market presence
business requirements exceed the Tier I functionality. delivering 24x365 services in a highly competitive
client-facing market space
Tier II is appropriate for firms such as n Businesses based on E-commerce, market transactions,
n Internet-based companies without serious financial or financial settlement processes
penalties for quality of service commitments n Large, global companies spanning multiple time
n Small businesses whose information technology zones where client access to applications and
requirements are mostly limited to traditional normal employee exploitation of information technology is
business hours, allowing system shutdown during a competitive advantage
off-hours
n Commercial research and development firms, such Companies who have extremely high-availability
as software, who do not typically have on-line or requirements for ongoing business, or for whom
real-time service delivery obligations there is a profound cost of disruption due to any data
center shutdown, select Tier IV site infrastructure.
These companies typically do not depend on real-time These companies will know the cost of a disruption,
delivery of products or services for a significant part of usually in terms of both actual dollar costs and impact
their revenue stream, or are contractually protected from to market share. The cost of disruption makes the case
damages due to lack of system availability. Occasionally for investment in high availability infrastructure a clear
companies will select Tier II infrastructure if they have business advantage.
become burdened with impacts due to nuisance equipment
outages associated with Tier I sties. A large number of
institutional and educational organizations select Tier II
infrastructure because there is no meaningful impact of
disruption due to data center failure. Some companies
have successfully used Tier II infrastructure to provide
off-site electronic vaulting for offline data.

11
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Making the Appropriate Tier Selection Another common oversight is the effect of shutting
Should Be Based on Business down electrical panels on the mechanical system the
Requirements panel feeds. If more than the redundant number of
Selecting the site infrastructure solution based on the chillers, towers, or pumps is de-energized for electrical
availability objectives required to sustain well-defined maintenance, computer room cooling is impacted.
business processes with substantial financial consequences
for downtime provides the best foundation for investment Occasionally, electrical systems fail to achieve Tier III or
in data center facilities. The owners focus during the data Tier IV criteria due to the UPS power distribution path.
center design and delivery process should be the consistent Topologies that include static transfer switches that cannot
application of the Tier Performance Standard, rather than be maintained without affecting computer room power, fail
allowing recurring debate over every characteristic or the concurrent maintenance criteria. UPS configurations
attribute that makes up the data centers site infrastructure. that utilize common input or output switchgear are almost
always often unmaintainable without computer room
Including criteria from a higher Tier classification, outages and fail the Tier III requirements even after
or an attribute leading to increased availability, does spending many hundreds of thousands of dollars.
not increase the overall Tier classification. Moreover,
deviation from the Tier standard in any subsystem Consistent application of standards is necessary
will prevent a site from classification at that Tier. For to have an integrated solution for a specific data
example, a UPS system patterned after a Tier IV system center. It is clear that the IT organization invests
within a site featuring a Tier II power distribution heavily in the features offered by newer computer
backbone will yield a Tier II site. The most significant equipment technology. Often, as the electrical and
deviations from the Tier Standard found in most sites mechanical infrastructures are defined, and the
can be summarized as inconsistent solutions. facility operations are established, there is a growing
degree of inconsistency in the solutions incorporated
Frequently, a site will have a robust fault tolerant in a site. As shown in Figure 3, each segment must be
electrical system patterned after a Tier IV solution, integrated to deliver the overall data center solution.
but utilize a Tier II mechanical system that cannot An investment in one segment must be met with a
be maintained without interrupting computer room similar investment in each of the other segments if any
operations. This results in the overall site achieving a of the elements in the combined solution are to have
Tier II rating. Most often the mechanical system fails effect on IT availability. A well-executed data center
concurrent maintenance criteria because of inadequate master plan or strategy should consistently resolve the
isolation valves in the chilled water distribution path. entire spectrum of IT and facility requirements.

Figure 3:
Comparing IT Solutions for Reliability, Availability, and Serviceability to Site Infrastructure

12
 The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Lifecycle Planning About the Authors

It is disappointing to observe brand new sites that Mr. Turner is a Distinguished Fellow and Senior
received very little thought during initial design to future Certification Authority for the Institute and a Principal of
operations. Valves were located in inaccessible places, ComputerSite Engineering, Inc. in Santa Fe, NM.
the access path for the addition of future components was Mr. Seader is a Distinguished Fellow and Certification
not thought out, or sufficient capacity to simultaneously Authority for the Institute and a Principal of ComputerSite
test new systems while sustaining the critical load was Engineering, Inc. in Santa Fe, NM.
not provided. These details could have been addressed Mr. Brill is the founder of the Institute and is its Executive
for no additional cost during design. This failure limits Director. He is a Principal of ComputerSite Engineering.
both investment value and site performance right from
its initial occupancy. A more sustainable site will project About The Uptime Institute
future requirements and anticipate them during the initial The Uptime Institute, Inc. is a pioneer in creating and
design and construction. operating knowledge communities for improving uptime
effectiveness in data center Facilities and Information
Sites should be designed to anticipate increasing power
Technology organizations. The 85 members of the
requirements and tier levels. These sites provide future
locations for necessary site infrastructure equipment as Institutes Site Uptime Network are committed to
well as a planned means to commission them and then achieving the highest levels of availability with many
connect them transparently to operational systems. being Fortune 100 companies. They interactively learn
from each other as well as from Institute sponsored
Institute Site Topology and meetings, site tours, benchmarking, best practices,
Sustainability Certification uptime effectiveness metrics, and abnormal incident
The Institute exclusively reserves the right to determine collection and trend analysis. From this interaction
Tier ranking and to certify sites as meeting Tier and from client consulting work, the Institute prepares
requirements as summarily described in this white paper. white papers documenting Best Practices for use by
This comprehensive process involves additional criteria Network members and for the broader uninterruptible
beyond the information provided herein. The process uptime industry. The Institute also conducts sponsored
is similar to that used for ISO 900X certification. The research and offers insightful seminars and training in
ISO standard is set and maintained by the International site infrastructure management.
Standards Organization who trains and certifies field
inspection agencies in different parts of the world. These
field inspectors inspect and validate conformance to the
ISO standard before certification is granted for a limited
time period. The Institute has licensed ComputerSite
Engineering Inc., a separate but related company,
to perform inspection and validation utilizing the
Institutes Tier Performance Standards and the Institutes
comprehensive database of emerging industry problems
and best design practices. Sites reviewed and certified
by the Institute can be seen at www.uptimeinstitute.org/
tui_certification.html.
1996, 2001-2006 The Uptime Institute, Inc.
Conclusion
Data center owners have the responsibility to determine
what Tier of functionality is appropriate or required for
their sites. As such, it is a business decision to determine
the Tier necessary to support site availability objectives.
Part of this decision is to balance the IT operational
practices with the facility practices that support the IT Building 100
2904 Rodeo Park Drive East Santa Fe, NM 87505
world. Once selected, however, the desired Tier should Fax (505) 982-8484 Phone (505) 986-3900
be uniformly implemented. tui@uptimeinstitute.org www.uptimeinstitute.org

TUI 705D
13
The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Illustrative Electrical System Topology - Tier I

Note: This diagram illustrates basic Tier I electrical distribution concepts. This
diagram shall not be interpreted to represent a standard or compliant electrical
system topology, or a solution fulfilling any particular set of requirements.

Site certification requires consistent application of Tier concepts to all 16 critical

subsystems that comprise data center site infrastructure.

14
The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Illustrative Electrical System Topology - Tier II

Note: This diagram illustrates a basic Tier II electrical distribution concept. This
diagram shall not be interpreted to represent a standard or compliant electrical
system topology, or a solution fulfilling any particular set of requirements.

Site certification requires consistent application of Tier concepts to all 16 critical

subsystems that comprise data center site infrastructure.

15
The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Illustrative Electrical System Topology - Tier III

Note: This diagram illustrates a Tier III electrical distribution concept. This
diagram shall not be interpreted to represent a standard or compliant electrical
system topology, or a solution fulfilling any particular set of requirements.

Site certification requires consistent application of Tier concepts to all 16 critical

subsystems that comprise data center site infrastructure.

16
The Uptime Institute
Tier Classifications Define Site
Infrastructure Performance

Illustrative Electrical System Topology - Tier IV

Note: This diagram illustrates a Tier IV electrical distribution concept. This

diagram shall not be interpreted to represent a standard or compliant electrical
system topology, or a solution fulfilling any particular set of requirements.

Site certification requires consistent application of Tier concepts to all 16 critical

subsystems that comprise data center site infrastructure.

17