DEV Community

Cover image for Immersion Versus Direct-to-Chip: The Heat Flux Economics of Liquid Cooling
gxqwb
gxqwb

Posted on

Immersion Versus Direct-to-Chip: The Heat Flux Economics of Liquid Cooling

Air cooling has a long and honourable history and it is ending. Not because fans stopped working, but because the density of heat coming off a package crossed a threshold where air can no longer carry it away without moving an unreasonable amount of it. The number that matters is heat flux: watts per square centimetre of die area, not watts per rack.

A modern accelerator dissipates well over a hundred watts per square centimetre at the die, and the hotspot is worse. Air has a low volumetric heat capacity and a poor convective coefficient, so removing that heat requires either a very large temperature difference, which the silicon will not tolerate, or a very high air velocity, which costs fan power and floor space. Somewhere around fifty watts per square centimetre, air stops being practical.

Above that line, two approaches compete, and they disagree about almost everything: what the coolant touches, what the operator can service, and what a datacentre is for.

Two answers to the same thermal problem

Direct-to-chip cooling puts a cold plate on the packages that need it and circulates water or a water-glycol mix through it. The rest of the board stays air-cooled. The thermal path is short, the coolant volume is small, and the retrofit is incremental: you change the cold plate and the loop, not the server architecture.

Immersion puts the entire server into a tank of dielectric fluid. Single-phase systems circulate the fluid to an external heat exchanger; two-phase systems let it boil and condense on a coil above the tank, which moves far more heat per unit of flow but requires a sealed vessel and a fluid whose boiling point matches the operating temperature.

The engineering difference is the thermal resistance chain. A cold plate has to conduct through a thermal interface material into a metal block and then into the fluid, and each interface adds resistance. Immersion removes the interfaces entirely: the fluid touches the package, the memory, the power stages, everything.

Direct-to-chip is a better cold plate. Immersion is a different datacentre, and the cost difference between those two statements is where the market is being decided.

Heat flux capability by cooling method, watts per square centimetre

The cost structure that decides it

Immersion has an operating advantage that shows up in the power bill. Because the whole board is cooled, the facility does not need the large air handlers that consume a significant share of total power in a conventional hall, and the waste heat comes off at a temperature high enough to be useful for district heating or to drive a heat pump efficiently.

It has a capital and operational disadvantage that shows up everywhere else. The fluid is expensive, the tanks are heavy, the servers have to be designed or modified for immersion, and service means lifting a dripping board out of a bath. Optical modules, connector plastics and some capacitors are not qualified for long immersion, which limits which hardware can go in.

Direct-to-chip has the opposite profile. It is cheap to retrofit, serviceable with ordinary tools, and compatible with almost any server. It also leaves a substantial amount of heat in the air, because memory, drives and power supplies are still air-cooled, so the facility still needs air handling, just less of it.

Total cost of ownership per megawatt by cooling approach

There is a third approach that gets less attention and may end up mattering more: cooling the package through the substrate rather than the lid. If the heat can be pulled out of the back of the die through a silicon path with a low thermal resistance, the cold plate no longer has to sit on top of a stack of interfaces, and the flux ceiling rises without changing the fluid. That would extend the life of direct-to-chip considerably, and it is the main technical reason the immersion case is not yet settled.

The number to watch is thermal resistance per unit area at the die, because every cooling decision downstream is a consequence of that one figure.

Why the split will not be resolved by efficiency

The efficiency comparison usually favours immersion by a few percent of total facility power. That is a real number at scale, and it is not large enough on its own to justify rebuilding a hall.

What actually decides the choice is the constraint the operator is under. If the constraint is floor space or water availability, immersion's higher density per square metre wins. If the constraint is time, because capacity is needed in nine months, direct-to-chip wins because it can be retrofitted into an existing shell.

There is a third consideration that rarely appears in comparisons: the failure mode. A leaking cold plate drips onto a server. A leaking immersion tank empties a bath onto a floor. Operators price those two risks very differently, and the pricing is rational, because the second is rarer but worse.

What the next five years look like

The plausible trajectory is coexistence rather than replacement. Direct-to-chip becomes the default for new high-density halls because it is the cheapest way to get from here to there, and immersion becomes the choice for specific sites where density, heat reuse or water constraints dominate.

Stated with reasoning: datacentre cooling equipment is currently a market in the low tens of billions annually, growing with capacity. If liquid cooling reaches the majority of new high-density capacity by 2030, the liquid-cooling segment alone lands between twelve and thirty billion dollars a year, with the range driven mostly by how quickly chip power rises past the point where a cold plate can no longer keep up without two-phase fluid.

The technology that wins will be the one whose service model the operators already know, because a cooling system is judged by what happens on the worst day, not the average one.

What is actually transferable

The pattern is a familiar one in infrastructure: two technologies solve the same physical problem with different answers to the question of where the boundary of the product sits. One extends the existing architecture; the other replaces it. The extension almost always ships first, and the replacement wins only where the extension runs out of room.

The same trade appeared between in-row cooling and room cooling, between AC and DC distribution, and between hard disks and tape in archives. In each case the incumbent architecture was extended until a physical limit made extension impossible, and only then did the replacement become economic.

For anyone specifying a facility today, the useful discipline is to compute the heat flux of the hardware you expect in five years and check whether your cooling path can carry it. Air will not. The interesting question is only which liquid gets there first.


If this saved you time, a tip is appreciated: HCqPmPJM3NVtS6YrmVaTZYnuP8XZ2PFgQZKT94VK1LkS

Top comments (0)