Power and cooling for GPU clusters
This is where many GPU projects stall. Modern GPU servers draw far more power than the racks most data centres were built for, and all of that power becomes heat. Checking the facility early saves months.
Power figures to plan with
| System | Approximate maximum power |
|---|---|
| 8-GPU A100 server (DGX A100 class) | ~6.5 kW |
| 8-GPU H100 or H200 server (DGX H100 class) | ~10.2 kW |
| 8-GPU B200 server (DGX B200 class) | ~14.3 kW |
| Rack-scale GB200 NVL72 system | around 120 kW per rack, liquid-cooled |
| Single L40S GPU | ~350 W |
Vendor maximum figures. Real draw under training load is close to the maximum, so plan with these numbers, not with averages.
A quick worked example
Four 8-GPU H100 servers draw about 41 kW. Add network switches, storage and management servers and the total is around 47 kW.
- In a data centre built for 8 kW racks, that is six racks mostly empty, and cooling may still not cope with the hot spots.
- In racks rated for 25 kW with rear-door heat exchangers, it fits in two racks.
- With direct liquid cooling, the same servers can be packed even tighter, but the facility needs water connections to the racks.
Cooling options
| Option | Practical rack density | Notes |
|---|---|---|
| Air with hot and cold aisle containment | Up to about 20 to 25 kW | Works for small clusters if the room has spare cooling |
| Rear-door heat exchangers | Roughly 30 to 50 kW | Chilled water at the rack door; servers stay air-cooled |
| Direct liquid cooling | 50 kW to 100 kW and more | Coolant to the chips; needed for the newest rack-scale systems |
Data-centre readiness checklist
- Power per rack available, and on which circuits (A and B feeds)
- Total spare power and cooling capacity in the room, not just per rack
- Floor loading: fully loaded GPU racks are heavy
- Chilled water availability if rear-door or liquid cooling is needed
- Space for cable trays: a 32-server cluster has hundreds of fabric cables
- Lead time for any facility upgrades, which is often longer than for the servers
More AI infrastructure guides: Training cluster · Inference platform · GPU network fabric · Storage for AI · FAQ · Use case: private LLM platform
Planning a GPU cluster?
Tell us the models you want to run or train, how many users, and where it will be hosted. We will come back with a first sizing: GPUs, servers, network, storage, and the power and cooling your data centre will need.