Welcome

Product Training – CRAC/CRAH/High-Density

General topics - Data Center…

Availability

Safety

Energy Efficiency

Transparency

http://www.stulz.de/en/newsroom/blog/the-data-center-standard-din-en-50600-in-brief-563/ 3
Air-Conditioning requirements in a Data Center

4
Air-Conditioning requirements in a Data Center

“Correct“ temperature and “correct“ air

volume in front of the IT-equipment –
24 hours a day and 365 days a year –
all that as energy-efficient and reliable
as possible and in dependence of the
actual heat load of the servers.

http://www.stulz.de/en/newsroom/blog/data-center-temperature-control-1467/ 5
Heat loads

High Density

Medium
Density
Low Density

< 7kW / Rack 7-15kW / Rack > 15kW / Rack

6
Requirements of the IT-equipment

ASHRAE TC9.9-2013

Recommended supply
air temperature range:
18°C up to 27°C

Allowable supply air

temperature range:
15°C up to 32°C

7
 ∆T – Airside temperature difference
 Generally speaking, the required supply air temperature must be reached at a
height of 1,5m in front of the rack
 The delta T across the rack is typically between 10K and 15K
 The air outlet temperature is normally not important for the IT equipment

Server Inlet Server Outlet

Temperature Temperature
1,5m

18 - 27°C 28 - 42°C

http://www.stulz.de/en/newsroom/blog/delta-t-91/ 8
∆T – Airside temperature difference (ideal)

Server Outlet = 39°C Return Air = 39°C

𝛥𝛥T = 15K 𝛥𝛥T = 15K

Server Inlet = 24°C Supply Air = 24°C

9
SHR

SHR = Sensible Heat Ratio (best achievable: 1,0)

Sensible Capacity
SHR= Total Capacity
Target: Sensible Capacity = Total Capacity
No unwanted, efficiency decreasing dehumidification

10
EER

EER = Energy Efficiency Ratio (the higher, the better)

Total Capacity
EER= Total Power Input
Lowest total power input for a specific cooling capacity

11
AER

AER = Airflow Efficiency Ratio (the lower, the better)

Fan Power Input

AER= Airflow
AER Unit: W / (m³/h)
AER applies at specific airflow and external static pressure

http://www.stulz.de/en/newsroom/blog/aer-351/ 12
PUE (pPUE, iPUE, dPUE)

PUE = Power Usage Effectiveness (the lower, the better)

Total facility power

PUE= IT power
KPI = Key Performance Indicator (EN 50600-4-2, “Green Grid”)
NO BENCHMARK!
NO EFFICIENCY FACTOR!
http://www.stulz.de/en/newsroom/blog/powerusageeffectivenesspueppue-1045/ 13
PUE (pPUE, iPUE, dPUE)

The PUE calculation is a Data Center specific measure to understand

and improve the energy consumption
The PUE is based on MEASUREMENTS of a period of at least 12
months and therefore a yearly average value!

Elec.
Electricity IT Energy
facilities
Electricity
Other Energy
Measuring point

Block heat
and power Cooling, Air-Conditioning, Lightning,
Controls,…
plant

Natural gas PUE = (1.450 + 450 + 400) /(1.500) = 1,53

Source: EN 50600-4-2

14
PUE (pPUE, iPUE, dPUE)

PUE “derivatives”:

“Interim” PUE = iPUE: period of measurements less than 12 months

“Partial” PUE = pPUE: consideration of a subset of the infrastructure

“Draft” PUE = dPUE: theoretical PUE based on a “real” DC planning

15
Products & Solutions
Mission Critical Cooling

17
Our vision

STULZ is the world's leading

solution provider of energy
efficient temperature and
humidity management
technology, specifically for
mission-critical applications.

18
Products

19
20
STATISTICS

 IHS “Data Center Cooling Report“ 2016 – Cooling Market (Revenue)

1% 1%
2015 2020
13% 14% 15% 14%
9%
13%

63% 57%

Chip/Server Row/rack Perimeter AHU Containment Chip/Server Row/rack Perimeter AHU Containment

21
STATISTICS

-4%
 EUROVENT 36,958 35,655 34,837 35,154
Market 32,866
Intelligence TLC 8,419 7,867 6,831
6,914 29,746
(Sold units 6,527
6,470
EMEA)
CW 11,143 11,481 10,708 11,446
10,758
9,684

DX 17,396 16,307 17,215 16,877 15,581 13,592

2011 2012 2013 2014 2015 2016

22
STATISTICS

 EUROVENT
Market
Intelligence 8,419
(Capacity EMEA)
CW 730k 790k 650k 760k
660k 730k

DX 580k 550k 590k 550k 520k 500k

2011 2012 2013 2014 2015 2016

23
ROOM COOLING – Products

24
ROOM COOLING - CyberAir3(PRO)

CyberAir 3(PRO)
• Cooling Capacity ~ 20kW – 250kW
• EC-fans
• 7 different cooling systems
• 7 unit sizes
• 4 different airflow directions (Downflow, Upflow,
Raised Floor, High)
• Different compressor types (Scroll, EC-Scroll,
tandem, one circuit, two circuits)
• Three different refrigerants available

25
CyberAir 3(PRO) – COOLING SYSTEMS

Air-Cooled System Chilled Water System Dual-fluid System Indirect Free Cooling
(A, AS) (CW) (ACW, ASCW) System
(GE, GES)

Water-Cooled System Dual-fluid Chilled Water Dual-fluid System

(G, GS) System (CW2) (GCW, GSCW)
Cooling Systems – Dual-Fluid Example

Standard daytime operation: Night-time, weekends:

 Chiller on  Chiller off
 Office A/C on  Office A/C off
 CW-circuit in ACW unit  DX-circuit in ACW unit
active active
 Compressor and  Compressor and
condenser off condenser on

http://www.stulz.de/en/newsroom/blog/dehumidification-in-data-centers-when-using-cw-units-at-high-temperature-levels-362/
 Cooling Systems – GE/GES

Regions for Free Cooling

Indirect Dynamic Free Cooling with GE and GES
units.

 Automatic choice of four different working modes

• DX (compressor cooling)
• Mix-Mode (Compressor and Free cooling)
• EFC (Extended Free cooling)
• FC (Free cooling)
MIX

EFC
DX

FC

28
Unit Sizes CyberAir 3(PRO)

Unit size 1 Unit size 2 Unit size 3

950 x 890 x 1980mm 1400 x 890 x 1980mm 1750 x 890 x 1980mm

950 x 890 x 2495mm 1400 x 890 x 2495mm 1750 x 890 x 2495mm
Number of fans: 1 Number of fans: 1 Number of fans: 2
All unit types/cooling systems All unit types/cooling systems All unit types/cooling systems

29
Unit Sizes CyberAir 3(PRO)

Unit size 4 Unit size 5

2200 x 890 x 1980mm 2550 x 890 x 1980mm

2200 x 890 x 2495mm 2550 x 890 x 2495mm
Number of fans: 2 Number of fans: 3
All unit types/cooling systems All unit types/cooling systems

30
Unit Sizes CyberAir 3(PRO)

Unit size 7 Unit size 8

3110 x 980 x 1980mm 3350 x 980 x 2495mm

3110 x 980 x 2495mm Number of fans: 4
Number of fans: 4 ASR/ASH…CW/CW2 only
All CW-units, ASR/ALR…DX

31
ROOM COOLING – Airflow directions

DOWNFLOW
ASD

UPFLOW
ASU

32
ROOM COOLING – Airflow directions

RAISED FLOOR
ASR

HIGH
ASH

33
Airflow directions / Available unit types

A G GE CW CW2 ACW GCW

AS GS GES ASCW GSCW

ASD X X X X X X X
(Size 1)

ALD - - X - - - -

ASU X X X X X X X
(Size 1)

ALU - - X - - - -

ASR X - X X X - -

ALR - - X - - - -

ASH - - - X - - -

34
CyberAir 3(PRO) – Main Features

 EC-fan technology for energy efficient air flow in all

operating points

 EC-compressor technology for infinite compressor

control with maximum efficiency and precise
temperature control

 Maximized heat exchanger and filter surface

 Quiet running, long life, maintenance friendly

CyberAir 3(PRO) – Main Features

 Compact dimensions

 High-efficiency zig-zag filters plus filter control

management

 Electronically controlled expansion valves

 Different coil lay-outs to meet all required operating points

 Stulz C7000 microprocessor control of all active

components
CyberAir 3(PRO) – Energy Saving Features

Latest EC-motor, electronics and fan blade

technology
Fan blade made out of fibre-glass reinforced
material
• Aerodynamically optimized profiles
• Noise optimized design
• Economization of weight
Fans especially developed for Stulz
CyberAir 3(PRO) – Energy Saving Features

Lowest fan power consumption in full

and part load
Very low noise emission
Maintenance free
Excellent performance data (EC- Stulz

Directive 2005/32/EC = Energy

related products directive „ErP“)
AC
EC
ErP 2013
ErP 2015
CyberAir 3(PRO) – Energy Saving Features

EC-Compressor:
Maximized efficiency especially in
part load operation and Mix-mode
Constant supply air temperature
Integrated soft-start
Fast and precise reaction according
to the actual heat load
Long compressor life time because
of continous compressor operation
without start- and stop-times

Page 39
CyberAir 3(PRO) – Energy Saving Features

Airflow optimisation:
Aerodynamic design for lowest air
side pressure drop
Perfect air flow through unit, filters
and coils from air inlet to air outlet
Size optimized heat exchangers,
filters and fans
Optimization of fan positioning
CyberAir 3(PRO) – Energy Saving Features

High Performance Filtration:

Use of „high-performance zig-zag-filters“ with large
filter surface for reduced fan power consumption and
higher net sensible cooling capacity
Filter control management
• Integrated intelligent airflow control
• Adaption of fan speed in dependence of actual filter
pressure drop
• Automatic indication of required filter change
• Protection against too low airflow and „hot spots“
• Constant airflow at any time and longer filter life-time
The underfloor fan section (CW)

Reduction of internal air pressure losses

Fan position ensures less turbulence and
misdirection of the air
Both the above result in reduced fan
consumption

ASD 2080 CW (coil A): ASR 2060 CW (coil A):

Size 7, 35.000 m³/h, ESP: 20Pa Size 7, 35.000 m³/h, ESP: 20Pa
• Fan power: 5,5 kW • Fan power: 4,0 kW
• AER: 0,16 • AER: 0,11

http://www.stulz.de/en/newsroom/blog/cracs-with-underfloor-fan-section-569/ 42
The underfloor fan section (DX)

Reduction of internal air pressure losses

Fan position ensures less turbulence and
misdirection of the air
Both the above result in reduced fan
consumption

ASD 712 AS: ASR 732 AS:

Size 4, 20.000 m³/h, ESP: 20Pa Size 4, 20.000 m³/h, ESP: 20Pa
• Fan power: 4,1 kW • Fan power: 3,6 kW
• AER: 0,20 • AER: 0,18

http://www.stulz.de/en/newsroom/blog/cracs-with-underfloor-fan-section-569/ 43
 CW-Standby-Management

OFF OFF ON ON ON ON

ON ON ON ON ON ON

 6x ASR 1710 CW (coil A)  6x ASR 1710 CW (coil A)

 Passive redundancy (4 units ON, 2 units OFF)  Active redundancy (6 units ON)

 Airflow: 4x 32.000m³/h = 128.000m³/h  Airflow: 6x 21.300m³/h = 128.000m³/h

 Fan power: 4x 4,5kW = 18,0kW  Fan power: 6x 1,6kW = 9,6kW
 Total sound pressure level: 63dB(A)  Total sound pressure level: 55dB(A)
 Waterside pressure drop: 70kPa  Waterside pressure drop: 40kPa

http://www.stulz.de/en/newsroom/blog/standby-management-for-cw-units-77/ 44
Option “Differential Pressure Management”

Differential Pressure Management…

 Permanent control of fan speed in
accordance of heat load
 Quick and precise reaction on even smallest
changes
 Always provision of the correct airflow
 No interference between pressure and
temperature
 Always constant supply air temperature and
correct air volume

45
Option “Pressure Independent Control Valve”

Pressure Independent Control Valves…

 combine the features of a dynamic pressure
control valve and a 2-way-control-valve
 ensure a steady and high valve authority
 ensure a precise and stable operation under
all load conditions
 ensure an easy installation and quick
hydraulic balancing
 reduce the commissionig and start-up times
 contribute to reduce energy costs of the
pumps

http://www.stulz.de/en/newsroom/blog/use-of-pressure-independent-control-valves-in-cw-units-1409/ 46
Option “Direct Free Cooling (FCP)”

Free Cooling Plenum (FCP)…

 Possibility of Direct Free Cooling in combination with
CRAC/CRAH
 No extra space needed (both dampers on top of the unit)
 Three different working modes:
• DX/CW mode
• MIX-Mode
• Free Cooling
 Frost protection included

47
CyberAir 3(PRO) – Other Features/Options

CyberAir 3(PRO) CyberAir 3(PRO)

• Dual power supply • Raised floor stands
• UPS buffer controller • Different filter classes, different coils
• Pressure Independent Control Valves • Mechanical devices (ducts, dampers, …)
• Re-heat, electrical, proportional, LPHW, • „Filter-Control-Management“
refrigerant • Differential Pressure Control
• Steam humidifier, control of external ENS • CW-Standby-Management
• BMS-connections • Direct Free Cooling (FCP)
• Monitoring (WIB) • Display of airflow, EER, cooling capacity
• …

http://www.stulz.de/en/newsroom/blog/cw-units-heat-exchanger-62/ 48
CyberAir 3(PRO) – Competition

…and many more…

http://www.stulz.de/en/newsroom/blog/cooling-capacity-how-to-compare-apples-with-apples-1400/ 49
CyberAir 3(PRO) – Competition

 At least 40 competitors producing CRAC/CRAH units worldwide that can be

grouped into 3 different parts:
• “Large Global Players“: Vertiv, Schneider, STULZ
• “Mini Global Players“: Climaveneta, Airedale, GEA Denco, Silent-Aire, Huawei
• “Local Players“: EMEA: WKT, HiRef, Tecnair, Aermec, RCGroup,…
APAC: Daikin, Citec, AIRSYS,…
LAM: Data Aire,…
 Three major tendencies are visible:
• Main competition (Vertiv, Schneider, Airedale) is focussing on systems
• Brochure data is published with higher return air and water temperatures
• More and more competitors with only little information
 “Smaller“ Italian manufacturers are publishing technically questionable data and are
still focussing on the product
 Tendency to capacity optimization and higher capacity per footprint
50
ROOM COOLING – CyberAir 3(PRO) USPs

 Flexibility: Airflow directions, cooling systems, refrigerants, coils, valves…

 State-of-the-art operating temperatures (air-side and water-side)

 Best “overall“ performance data available on the market (EER, AER)

 Only manufacturer with a full DX-range with fan section underfloor

 Filter-Control-Management, Differential Pressure Control

 World-wide number one indirect free cooling system

51
ROOM COOLING – MiniSpace EC

MiniSpace EC
• Cooling Capacity 5kW – 25kW
• For smaller Data Centers
• 3 Cooling Systems (A, G, CW)
• Upflow, Downflow, Displacement
• EC-fans
• Direct Free Cooling possibility with EcoCool

52
ROOM COOLING – MiniSpace EC

MiniSpace EC
 Precise, reliable and cost-effective air-
conditioning solutions
 Compact size – MiniSpace EC requires very little
floor space
 Easy integration in small to medium-sized server
and technology rooms
 Installation in existing server rooms with no
problem

53
ROOM COOLING – CyberLab

CyberLab – ASU 211 AL

• Cooling Capacity up to 20 kW
• “0-heat-load-unit“
• According to German hygienic standard
VDI6022
• Very precise controls
• Easy access to all components
• EC-fan, EC-compressor
• Standardized solution

54
Products

55
HIGH DENSITY – Products

56
HIGH DENSITY – CyberRow

• Targeted cooling of high(er) density racks

• Independent of rack manufacturer
• To be positioned directly between or next to the racks
• Suitable for applications with or without raised floor
• Suitable for use in open or contained hot-aisle/cold-
aisle applications
• Infinitely variable cooling capacity
• Cooling Capacity 20 kW – 58 kW
• 3 unit sizes (300/400/600mm)
• 4 Cooling Systems (A/AS, CW, GS, GES)

57
HIGH DENSITY – CyberRow

Air velocity  Special airflow deflectors between the fans

plus airflow straighteners at the discharge:
• To increase the maximum fan throwing
range
• For an even and horizontal airflow
• For a reduced fan power consumption
 Thanks to the horizontal (lateral) air
discharge, CyberRow creates uniform and
close-contact airflow directly in front of the
racks.
In most cases therefore a constructional
partition of hot aisle and cold aisle can be
avoided.

58
HIGH DENSITY – CyberRow Cooling Systems

• Chilled Water (CW) • Air-Cooled (A/AS)

• CRS/L 210/320/560 CW (300/400/600mm) • CRS/L 211/251/361 A/AS (300/400/600mm)
• Proportional controlled 2-way-valve • Either stepless controlled EC-compressor or
• 2 different coils per size on/off compressor with hot-gas-bypass
• R410A or R134a
• Oil separator for AS-units

59
HIGH DENSITY – CyberRow Cooling Systems

• Glycol-cooled (GS) • Indirect Free Cooling (GES)

• CRS/L 251/361 GS (400/600mm) • CRS/L 211/251/361 A/AS (400/600mm)
• EC-compressor, R410A • EC-compressor, R410A
• Plate condenser and 2-way-valve • The only (feasible) indirect free cooling
row-cooling unit available on the market

60
HIGH DENSITY – CyberRow Controls
CW valve, compressor speed
 6 temperature sensors for a close and
100%
energy efficient cooling control in 3
independent horizontal zones (full load and
16% part load)
0% • 3 supply air temperature sensors
Max (TIx1, TIx2, TIx3)
Setpoint supply air • 3 return ar temperature sensors
 2 diverse set-points (parameter) for a fast
Fan x speed and precise reaction on all requirements
max • Fan speed control in dependence of temperature
difference between return and supply air
• Compressor speed (valve opening, hot-gas-
min bypass control) in accordance of supply air
temperature
TRx - TIx
Setpoint delta T

61
HIGH DENSITY – CyberRow Airflow

Horizontal discharge for Front discharge for Cold Horizontal discharge for
Open Aisle Applications Aisle Containment Hot Aisle Containment

Increasing of efficiency/redundancy in most cases only possible with containment systems.

62
 HIGH DENSITY – CyberRow Dimensions

Two different unit depths to meet all requirements

a) 1.175/1.200mm (CRS = Standard) b) 1.375mm (CRL = Large)
CyberRow CRS

CyberRow CRS
Approx. 1.000mm

CyberRow CRL
Approx. 1.200mm
Approx. 1.200mm
RACK

RACK

RACK
63
HIGH DENSITY – CyberRow Options

 Supply air discharge through the front (400/600mm)

 Side flow – left or right side only
 Dual power supply
 2kg/h steam humidifier (400/600mm)
 Electrical re-heat (400/600mm)
 Humidity sensor
 Condensate pump
 Water detection system
 Supply air sensors to be positioned directly in front of the rack
 Connection to BMS systems

64
HIGH DENSITY – CyberRow Competition

 In general the market for “Row-Cooling“ can be devided into three different chapters:
• “Stand-alone“ row-cooling CW
• “Stand-alone“ row-cooling DX
• “Close-Loop“ cooling solutions (e.g. Micro DC) with similar cooling equipment
 3 main competitors: Rittal, Schneider (APC), Vertiv (Knürr)
• Rittal is focussing on “Close-Loop“ and “stand-alone“ CW
• Schneider (APC) is focussing on “Hot-Aisle-Containment“ plus “stand-alone“ CW
• Vertiv (Knürr) is focussing on “Close-Loop“ CW and “stand-alone“ CW
 Other competition either “Close-Loop“ (e.g. Minkels) or “stand-alone“ (e.g. Climaveneta)

 In most cases the sales channel is different compared to CRAC/CRAH

 The majority of the projects is via “rack“ or “cabling“ – cooling not important…

65
 300mm CW-version – Vertiv, Rittal, Schneider

ENP Rittal Schneider

38°C 14°C 44°C 21°C 35°C 17°C

7°C 12°C 9°C 16,3°C 10°C 15,5°C

Airflow: 7.800m³/h Airflow: 8.000m³/h Airflow: 7.200m³/h

Capacity: 59,5kW Capacity: 55,0kW Capacity: 39,4kW
Fan power: 2,5kW Fan power: 2,8kW Fan power: 1,8kW
EER: 23,80 EER: 19,64 EER: 21,89

Airside delta T: 24K !!! Airside delta T: 23K !! Airside delta T: 18K!

http://www.stulz.de/en/newsroom/blog/delta-t-91/ 66
HIGH DENSITY – CyberRow USPs

 Control features (2 independent set-points,

Delta T control, quantity of temperature
sensors)
 Flexibility (3 different sizes, different unit
depths, different airflow directions)
 Only manufacturer with a 400mm version
 Only manufacturer with a reliable and
good indirect free cooling solution
 “quasi-monopoly“ in terms of DX: number of
units, refrigerants, compressor types,…
 Energy efficiency (fan power consumption)
 Realistic, practice orientated air-side
temperature differences

67
HIGH DENSITY – CyberRack Active Rear Door

• Containment at rack level (instead of

room level or for supplemental cooling)
• “Active Rack Cooling“
• Cooling exactly where and when needed
• For network and server cabinets with
passive or active ventilation
• Standard racks can be converted into
high-density racks
• Components: EC-fans, CW heat
exchanger, controller

68
HIGH DENSITY – CyberRack Active Rear Door

• Complete product consists of heat exchanger

module and connecting frame
• Independent from rack manufacturer (adapter
frame needed for 600mm or 800mm rack,
drawing required)
• 2 different capacity versions (19kW/32kW)
• Dual power supply integrated
• Plug connections

69
HIGH DENSITY – CyberRack Active Rear Door
25°C 25°C

Air discharged by the servers is immediately Server Server

sucked in by the cooling unit and delivered back 40°C 40°C
45° 45° WT,in:14°C
into the room after treatment C C
WT,out:19°C
Therefore: 24°C 24°C

 no need of containment to separate hot and 25°C 25°C

cold air
 higher intake air temperature Server Server
 higher water temperatures 40°C
40°C
45° 45°
 high efficiency C C
WT,in:14°C

WT,out:19°C
24°C 24°C

70
 HIGH DENSITY – CyberRack Active Rear Door

Application 1 Application 2 Application 3

Rooms w/o Raised Floor: Data Center: High Density Data Centers:
 CyberRack to be used as  As a “cooling booster“ in  Individual areas (or
a stand-alone cooling unit combination with CRAC/ complete DC) with high-
in small server rooms CRAH to avoid hot-spots density racks only
 Entire cooling capacity  In case the perimeter  No risk of hot-spots
provided cooling has reached  Almost no extra space
limits needed for cooling units

71
HIGH DENSITY – AirBooster/AirModulator

Airflow Management in Raised Floor Computer Room Environments

72
HIGH DENSITY – AirBooster/AirModulator

Airflow Management in Raised Floor Computer Room Environments

• Elimation of hot-spots
• increase of efficiency

1. AirModulator
2. AirBooster
3. Pressure Control Module
4. Differential Pressure
Control

http://www.stulz.de/en/newsroom/blog/increased-efficiency-thanks-to-raised-floor-grilles-with-adjustable-opening-angle-818/ 73
HIGH DENSITY – AirBooster2

AirBooster2
• Fan assisted floor tiles to increase the local airflow at
the intake area of the server rack
• Requirement: sufficient airflow provided by the
CRAC/CRAH units
• Up to ~4.400m³/h (75W @ 2.900m³/h)
• In two zones individual adjustable guide vanes
• 3 air temperature sensors, option pressure control
• Light and heavy duty grille
• BMS connection Modbus RTU
• Available with or without controller

74
HIGH DENSITY – AirModulator2

PCRAC
PPCMP
AirModulator2
• Variable air volume damper to enhance efficiency
• Requirement: sufficient airflow and pressure provided
by the CRAC/CRAH units
• Automatically adjustable damper, Servomotor with
spring return
• Low pressure drop due to large damper surface
• Low leakage rate
• 3 air temperature sensors, option pressure control
• Light and heavy duty grille
• BMS connection Modbus RTU
• Available with or without controller
75
THANK YOU