Data center cooling systems are the mechanical and fluid infrastructure that removes heat from servers and network equipment, keeping hardware within safe operating temperatures. Without them, a modern server rack would thermally shut down within minutes. How data centers are cooled depends on power density, climate, budget, and the workloads running inside, and those variables have shifted dramatically as hyperscaler AI infrastructure spend has pushed rack power from 10-15 kW toward 50-100 kW and beyond.
Why Cooling Is the Central Engineering Problem in AI-Era Data Centers
A traditional enterprise server rack draws somewhere between 5 kW and 15 kW. An H100-based AI training cluster can push a single rack past 80 kW, and dense GPU configurations from NVIDIA’s Blackwell platform are designed around rack densities exceeding 100 kW. Heat output scales directly with power consumption, so every watt of compute that enters a rack must leave as heat. The only question is how.
The physics set a hard ceiling. Air has low heat capacity compared to water, so at roughly 30-40 kW per rack most facilities hit the limit of what air can remove without extreme fan power and airflow velocity. Cooling typically accounts for 30-40% of total facility energy in sites without free cooling, which means understanding AI data center power demand is inseparable from understanding how the heat leaves. Operators who get cooling architecture right cut operating costs and improve sustainability metrics at the same time.
Air-Based Cooling: CRAC Units, CRAH Units, and Containment Strategies
Air cooling remains the dominant approach in most existing facilities and will continue to serve lower-density workloads for years. The two primary active cooling units are Computer Room Air Conditioning (CRAC) units and Computer Room Air Handlers (CRAH) units.
A CRAC unit runs a self-contained direct-expansion refrigeration cycle, requiring no central chilled water plant. This makes them simple to deploy in smaller facilities or edge locations, though on-off compressor cycling wastes energy at part load. A CRAH unit is a precision air handler connected to a centralized chilled water plant, where large chillers handle refrigeration more efficiently than many small compressors. Most hyperscale facilities use CRAH units on a chilled water loop for this reason.
Beyond the units themselves, containment architecture determines how effectively cold air reaches servers and how efficiently hot exhaust is removed. The two primary approaches are:
- Hot aisle / cold aisle arrangement: Server racks face alternating directions so that air intake faces of servers face a cold aisle, supplied with cool air from floor tiles or overhead ducts, while hot exhaust exits into a hot aisle. This simple arrangement reduces recirculation and improves cooling efficiency substantially compared to random rack placement.
- Hot aisle containment or cold aisle containment: Physical barriers, curtains, or hard enclosures isolate either the cold supply air or the hot return air. Containing the hot aisle and returning it directly to the CRAH unit prevents hot exhaust from contaminating the cold air supply, typically improving cooling efficiency by 20-30% versus open floor arrangements.
- In-row cooling: Cooling units are placed directly between server rows, reducing the distance cool air must travel before entering servers. This approach suits high-density rows where floor-based airflow cannot deliver sufficient volume at adequate temperature.
For more detail on how facilities combine these methods in practice, the full guide to air cooling methods covers configurations, airflow calculations, and real deployment examples.
Economization and Free Cooling: Airside and Waterside Economizers
Free cooling, technically called economization, is any strategy that uses outdoor air or water temperature conditions to partially or fully cool the facility without running mechanical refrigeration. When outdoor conditions permit, economization is the most cost-effective cooling mode available.
Airside economizers route outside air directly or indirectly into the cooling stream. A direct airside economizer pulls outdoor air into the data floor when temperature and humidity fall within acceptable ranges; an indirect version uses a heat exchanger to transfer cooling energy without mixing the two air streams, protecting against outdoor contaminants. Hyperscale sites in Scandinavia and the northern United States run on airside free cooling for a large fraction of annual hours.
Waterside economizers cool the chilled water loop via outdoor air through evaporation or dry heat exchange. When outdoor wet-bulb temperature drops far enough, the economizer can replace mechanical chiller operation entirely. Microsoft and Meta have deployed waterside economization widely across their campuses. Evaporative cooling pre-cools air or water through evaporation, highly effective in dry climates but at the cost of increased water consumption. That tradeoff between energy efficiency and water use is the central planning tension for operators in warm, arid regions.
Liquid Cooling: Direct-to-Chip and Immersion Systems
Liquid cooling brings coolant into direct contact with heat sources. Water carries heat roughly 3,500 times more efficiently per unit volume than air, which is why it opens a path to rack densities air cannot support.
Direct-to-chip liquid cooling, also called cold plate cooling, attaches a liquid-carrying metal plate directly to the processor package. A water-glycol mixture flows through the cold plate, absorbs heat, and carries it to a heat exchanger outside the rack. Residual air cooling handles the remaining heat from memory and storage components. NVIDIA’s H100 and H200 platforms support cold plate configurations, and most purpose-built AI accelerator servers now ship with direct liquid cooling options.
Immersion cooling submerges servers in a dielectric fluid. Single-phase immersion circulates the warm fluid to an external heat exchanger. Two-phase immersion uses a fluid with a low boiling point that vaporizes on hot components, rises, condenses on cooled coils, and falls back down. Two-phase systems handle rack densities well above 100 kW, making them the current leading approach for the most thermally demanding AI training environments.
Both approaches are evolving rapidly. Dedicated guides cover each technology in depth. The key decision here is whether your rack density and vendor support make liquid cooling necessary at all, which the final section addresses directly.
Heat Rejection: Chillers, Cooling Towers, and Dry Coolers
Every cooling system must reject heat to the environment. That equipment, sitting outside or on the roof, represents significant capital and operating cost.
- Chillers use a vapor-compression refrigeration cycle to cool water to temperatures in the range of 6-12 degrees Celsius for distribution to CRAH units or direct liquid cooling systems. Large centrifugal chillers, made by manufacturers including Carrier, Trane, and Johnson Controls, achieve high efficiency at full load but require careful sequencing across multiple units to maintain efficiency at partial load.
- Cooling towers reject heat from the condenser water loop by evaporating a small fraction of the water into the outdoor air. They are highly effective at lowering water temperature but require water treatment to prevent scaling, biological growth, and corrosion. Water consumption and legionella risk management are ongoing operational considerations.
- Dry coolers (also called fluid coolers when used in closed-loop mode) reject heat entirely through sensible heat exchange with outdoor air, consuming no water. Dry coolers are less thermally efficient than evaporative towers in warm conditions but are increasingly preferred in water-constrained regions, and they pair well with higher-temperature liquid cooling systems where coolant return temperature can be 40-50 degrees Celsius rather than 30-35 degrees.
Efficiency Metrics: PUE and WUE
Power Usage Effectiveness (PUE) is the ratio of total facility power to IT equipment power. A PUE of 1.0 is physically impossible, but modern hyperscale facilities routinely achieve annual averages below 1.2. Older enterprise facilities often land between 1.5 and 2.0, meaning cooling and power overhead consume as much energy as the servers themselves. The Uptime Institute publishes annual surveys tracking industry-wide PUE trends.
Water Usage Effectiveness (WUE) measures liters of water consumed per kilowatt-hour of IT load. Facilities leaning on evaporative cooling carry higher WUE even with strong PUE. A facility in Ireland running airside free cooling can achieve outstanding numbers on both metrics. A facility in Phoenix achieving low PUE through evaporative towers may consume thousands of liters of water per hour in summer. ASHRAE‘s A1-A4 thermal envelope classifications define server inlet temperature and humidity ranges; recent updates allow higher inlet temperatures, expanding free-cooling hours and reducing mechanical refrigeration dependence.
Choosing a Cooling Strategy by Power Density and Climate
No single cooling architecture fits every facility. The practical decision depends on three factors: rack power density, local climate, and whether you are building new or retrofitting.
For racks below 20 kW, well-implemented air cooling with hot aisle or cold aisle containment and a modern CRAH-plus-chiller plant is fully adequate and operationally familiar to most facilities teams. The capital cost is lower and the maintenance ecosystem is mature.
From 20 kW to roughly 50 kW, in-row cooling and rear-door heat exchangers extend the ceiling of air-based approaches. Some facilities layer direct liquid cooling onto specific high-density rows while keeping the rest of the floor on air, a hybrid model that limits the capital disruption of transitioning the entire infrastructure.
Above 50 kW per rack, direct-to-chip liquid cooling is effectively required for sustained operation. Immersion becomes the preferred architecture past 80-100 kW per rack, particularly when construction economics favor a clean-sheet design over retrofitting a raised-floor facility.
Climate compounds these choices. A facility in Helsinki can achieve annual average PUE below 1.1 purely on airside free cooling. A facility in Singapore operates near-year-round above the free cooling threshold, making mechanical chilling unavoidable and placing a premium on chiller efficiency.
The U.S. Department of Energy’s Better Buildings initiative documents real-world cooling improvements across hundreds of facilities, providing benchmarks for what is achievable at different building types and climate zones.
Frequently Asked Questions
How are data centers cooled?
Data centers are cooled by removing heat from servers through air or liquid cooling systems. Air cooling uses CRAC or CRAH units to circulate conditioned air through hot and cold aisles. Liquid cooling routes coolant directly to processors via cold plates or submerges hardware in dielectric fluid. The removed heat is ultimately rejected to the outdoor environment via cooling towers, chillers, or dry coolers.
What is the most efficient data center cooling system?
For low-density racks, airside free cooling in a cold climate achieves the lowest energy overhead, often reaching PUE values below 1.15. For high-density AI workloads, direct liquid cooling or immersion cooling removes heat more efficiently than air and allows warm water rejection at higher temperatures, which enables efficient dry cooler operation with lower mechanical refrigeration hours.
What is the difference between air and liquid cooling in data centers?
Air cooling circulates chilled air through the server room to carry heat away from equipment; it is simple, familiar, and adequate up to roughly 20-30 kW per rack. Liquid cooling delivers coolant directly to heat sources via cold plates or immersion tanks, handling densities above 50 kW per rack that air cannot manage. Liquid cooling requires more upfront infrastructure but achieves higher heat transfer per unit volume.
Why do AI data centers need better cooling?
GPU clusters for AI training and inference draw far more power per rack than traditional servers. A single rack of NVIDIA H100 servers can exceed 80 kW, compared to 10-15 kW for standard compute. Air cooling cannot remove that much heat at acceptable airflow velocities without massive fan power and noise. Direct liquid cooling or immersion cooling becomes necessary to sustain GPU performance without thermal throttling.
What is a good PUE for a data center?
A PUE below 1.5 is the minimum standard for a reasonably efficient modern facility. Purpose-built hyperscale campuses with free cooling regularly achieve annual averages below 1.2, with some Nordic facilities approaching 1.05. Older enterprise data centers commonly run between 1.6 and 2.0, meaning cooling and power overhead equal or exceed the IT load itself. PUE alone does not capture water consumption, so evaluate WUE alongside it.
