What a Smart Rack Valve Can Prove, and What It Cannot

The rack flow is on target. One branch can still be short of coolant. See what the valve's control mode hides, how far to trust its kilowatt reading, and which evidence belongs beside it.

The rack looks ready. Its return valve reports 180 liters per minute, a 10 K temperature rise and 120 kW of heat in the coolant. For a 120 kW rack of liquid-cooled GPUs, those are reassuring numbers. They match the design and may well be accurate.

In the worked rack below, the same display survives a restriction in one of its eight branches. That branch gets roughly half its usual flow. The rack total stays on target because the valve opens further and the other branches take more.

The valve is doing its job. That is precisely why the screen needs a closer look: a controller can hide a disturbance by correcting for it. A film on a cold plate or a loss of corrosion inhibitors needs evidence from elsewhere altogether.

Before using those readings to accept a rack or enforce a limit, separate three things: what the valve measures, what it calculates, and what its control mode keeps steady. The distinction changes what to check next.

What the valve measures, and what it works out

A smart rack valve combines a control valve, a flow meter, two temperature sensors and a small controller. One example with detailed public documentation is Belimo's Energy Valve, offered for direct-to-chip cooling. Its datasheets describe ultrasonic flow measurement "with glycol and temperature compensation" and two Pt1000 sensors, one on a cable so it can sit on the other pipe.

Everything else is worked out or added. Heat is calculated from flow and the temperature difference, using the density and specific heat of the fluid the valve is set for, a setting the operating manual calls the medium and also uses to calculate flow.

Valve position comes from the actuator. Differential pressure needs a separate sensor, which the datasheet says is not included. With it, the controller can hold differential pressure, an option the maker describes for direct-to-chip racks, as well as position, flow or power. The controller moves the valve to bring its chosen value toward the setpoint. Available pressure, valve travel and control limits decide whether it gets there.

What a held setpoint hides

Start with the number the valve is trying to hold. In flow mode, extra rack resistance makes the valve open further. Flow can stay on target while valve position and the distribution inside the rack change. Looking only at the controlled value misses the work being done to keep it there.

Real control is not exact. One of the maker's datasheets gives its flow control accuracy as ±5%, or ±10% with glycol. Readings also move while the controller responds. Once conditions settle, a near-target flow can still hide extra resistance if enough pressure and valve travel remain to compensate for it.

The manual describes that point in each mode. In flow mode, a fully open valve still 5% short of its setpoint is logged in the status summary. In pressure mode, a valve that cannot reach its setpoint goes to maximum opening.

Flow mode versus pressure mode

The figure runs five events through one assumed rack, not a real product or site: the opening's 120 kW and 180 L/min across eight identical branches, one per tray. That flow is 1.5 L/min per kW, the minimum design flow in NVIDIA's DSX reference design.

The coolant is PG25-type (water with about 25% propylene glycol by volume, plus corrosion inhibitors). Branch pressure drop rises with the square of flow, and the valve holds its setpoint exactly, with no tolerance band.

In flow mode, restricting one branch to about half its normal flow leaves rack flow, rise and heat where they were, because the other seven branches each take a little more. Coolant in the restricted branch now warms by almost 19 K instead of 10 K. The valve notices only that it opened further and, if a pressure sensor is fitted, that the rack pressure difference rose.

In pressure mode the same restriction does show up: rack flow falls about 6% and the rise grows slightly, while heat holds steady. A removed tray looks different, with flow and heat both down by one eighth and the rise unchanged. Read together, the three readings tell the two events apart.

That tray case is the maker's own argument for pressure mode in direct-to-chip racks: the remaining cold plates keep their flow instead of sharing out the removed server's.

The useful clue is the combination, not a single alarm. Flow and heat falling together point somewhere different from flow falling while heat stays put. Even then, neither mode identifies the branch or sees a film on the cold plates. A warmer supply raises both measured temperatures without showing the lost margin at the chips.

In every tray and restriction case here, valve position moved. Trend it beside the number being held steady.

Putting an error bar on the kilowatt figure

Now look at the precision of that 120 kW. The display may show a neat number, but heat inherits the errors of flow and temperature. A current datasheet from the maker puts flow error at ±2% with water and ±5% with glycol, and temperature-difference error at ±0.22 K for a 10 K difference. It gives no combined figure for heat and nothing below 5 K. Other families and revisions differ, so use the sheet for your model.

At half load and the same flow, the assumed rack's rise halves to just under 5 K (from Dow's property table for DOWFROST LC 25), the edge of what the datasheet covers. The temperature error shrinks far less than the rise, so between a 10 K and a 5 K difference its share of the reading grows from 2.2% to 3.4%.

Combining the two errors takes an assumption. The GUM, the international guide to measurement uncertainty, combines independent standard uncertainties as the square root of the sum of their squares. Applying that arithmetic directly to datasheet limits is a sensitivity exercise, not a calibrated uncertainty interval or a stated confidence level.

If both errors sit at their limits in the same direction, they simply add, which is how the heat meter recommendation OIML R 75 sets a complete meter's maximum permissible error. For the glycol rack at full load, the two methods give roughly ±7 to ±9 kW on 120 kW.

A wrong fluid setting adds an error that never averages out. A liter of water carries about 3% to 4% more heat per kelvin than a liter of LC 25 between 30 and 50 °C (NIST data against Dow's table), so a meter left on water in a PG25 loop reads that much too high, before any effect on its flow reading.

Compare heat readings only at similar load and flow, where they carry about the same error, and lean on trends: another of the maker's datasheets quotes ±0.5% flow repeatability, so a stable installation can show flow changes smaller than its stated accuracy, provided the meter and its settings have not changed.

The number is also heat that reached the liquid, not the rack's electrical power. ASHRAE TC 9.9 asks designers to understand the heat capture ratio, because some IT power leaves through air.

When the heat reading steers the valve

Some smart valves can also act on their heat reading. The manual describes a power control mode and a power limit that can sit on top of pressure control, and the maker's product information adds that once the calculated power reaches the limit, the valve opens no further.

Here the distinction stops being just a measurement question. On a chilled-water coil, closing the valve reduces the heat moved. On a cold plate, the chips set the heat. While they hold their power, nearly all of it enters the coolant at whatever flow arrives. Cutting flow does not lower that heat. It raises the coolant temperature.

That changes what a power limit does. With an accurate reading, a limit that acts only above its threshold stays inactive at or below that threshold. A return sensor that reads high adds an error proportional to flow. Reducing flow can shrink that error without removing any of the chips' heat.

The next figure assumes a controller that actively closes the valve above the limit, with no tolerance band. That is different from the documented rule that simply prevents further opening. It compares settled readings at constant chip power, not a restart or a prediction of a real controller's timing.

The rack would take 180 L/min in pressure mode without the assumed pull-back limit. Its limit is set at the 120 kW design load.

With 117 kW in the coolant and the return sensor reading 0.4 K high, the valve closes until its reading meets the limit, at about 112 L/min. The chips still deliver 117 kW, so the coolant warms by almost 16 K instead of under 10 K.

A cap that only prevents further opening could encounter the same crossing while opening, if heat remains steady and the readings settle at each step. That calculation does not show whether it will reach that point during a real transient, or make it pull back from an already open position.

That offset is nearly twice the datasheet's ±0.22 K at 10 K, so the sensor pair would be out of specification. An offset within specification does the same closer to the limit, if the limiter has no tolerance band: at 118 kW with the return reading 0.2 K high, flow settles near 150 L/min and the rise grows by about 2 K.

At exactly 120 kW of true heat and a 120 kW limit, a correct sensor reports the limit at every flow: this controller has no reason to close. With a positive sensor offset, every positive flow reads above the limit. If true heat exceeds the limit, an accurate sensor also leaves no satisfying flow. The assumed controller then keeps closing toward its minimum. A real rack may reach a temperature or protection limit first; NVIDIA's management library identifies thermal slowdown events, but this steady-state model does not predict when they occur. A sensor that reads low can instead hide heat above the limit.

Four things the valve cannot see

The valve's view ends at its own pipe, so these four need evidence from elsewhere. Our article The GPUs Are Hotter. Is the Cooling Loop to Blame? maps what each measurement location can tell you.

Coolant condition

Beyond a glycol concentration estimate, if that option is active, the valve sees nothing of the coolant's chemistry. Concentration alone does not show inhibitor health, as Inside PG25 explains. The Open Compute Project (OCP) PG25 guidelines ask for each loop to be lab tested at least yearly, with in-line sensors helping between samples, and list what each sensor misses.

Reliability Engine's Virtual Chemist fills part of that gap: a side-stream device trends conductivity, glycol concentration, pH, a local corrosion trend and turbidity, while laboratory testing remains part of the program.

Distribution between branches

A rack valve measures the total, and the maker's own direct-to-chip flyer acknowledges that managing total flow does not by itself ensure equal distribution to each cold plate. That needs branch flow or temperature where fitted, compared with the rack's accepted baseline and with peer racks at similar load.

Heat transfer at the cold plate

A thin film can make the chip hotter while rack flow and coolant temperatures stay unchanged. That comparison holds the chip's heat input and the flow steady, with no added hydraulic restriction. If deposits also restrict a passage, or the GPU reduces power as it warms, the valve readings can change too. OCP's cold plate test specification calculates thermal resistance from the cold plate's base temperature, the coolant inlet temperature and the power, so tracking it in service needs chip-side temperatures. The 0.1 mm Heat Tax covers the mechanism.

Useful GPU output

Lower heat at the valve looks the same whether the workload eased or the GPUs slowed down to protect themselves. GPU telemetry, including thermal clock event reasons, separates the two.

Virtual Chemist's software puts available context from NVIDIA's Data Center GPU Manager (DCGM) in the same view as rack flow, temperature and pressure, with pump and valve state. It uses valve state as context and does not control the valve.

What to record beside every valve reading

For the next review, choose one rack whose valve readings already decide something, such as acceptance or a power limit. Keep these five items with the trend:

• The control mode and setpoint, plus any flow or power limit and why it was set.
• The fluid setting, checked against the loop's coolant, and whether glycol estimation is active.
• The temperature difference at the time, so the heat figure carries its error bar.
• Valve position and any logged saturation events.
• GPU load and any thermal clock event reasons over the same period.

A reviewer should be able to tell whether the rack changed, the controller compensated, or the measurement setting changed. Those five items make that possible. The valve then provides useful evidence about its part of the loop, without being asked to certify the parts it cannot see.