A Stable Temperature Can Hide a Slow Cooling Response
AI workloads can change heat load faster than air and liquid systems settle.
A rack may look acceptable at a steady workload yet lose margin when accelerators step into sustained training or inference demand. The commissioning question is therefore not only the final temperature; it is how inlet temperature, fan command, actual RPM, pressure, flow and component temperature move between the initial command and the new stable state.
ASHRAE recommends intelligent controls, granular rack-inlet monitoring and continuous commissioning for AI data centers. That supports a practical rule: capture the actual physical response, not only the BMS or BMC command that says a fan, damper or pump should have moved.
The failure mode has a name in operations: cooling capacity that arrives late. A rack reading 24 C at idle and 27 C at steady peak can still spend two minutes above its intended inlet limit while those two states trade places. A steady-state acceptance record contains nothing about that interval, and the interval is where accelerator throttling and thermal cycling actually occur.
Why the Power Step, Not the Plateau, Is the Hard Test
The transient is where control architecture, sensor placement and actuator speed are actually tested.
Step load is standard practice in electrical acceptance testing, and the same logic transfers to thermal acceptance. Published AI commissioning procedures apply step loads of roughly 0 %, 50 % and 100 % and observe how quickly the system recovers, with hold points and limits defined before the test begins. The thermal equivalent records how the cooling chain responds when the IT load arrives as a step rather than a gentle ramp.
Two effects make the cooling side harder. First, thermal mass stores heat before any sensor sees it: junction, package, cold plate, heat sink, air and coolant all lag in sequence, so the temperature a controller reacts to is already a delayed version of the load. Second, control loops tuned for the gradual profiles of conventional enterprise compute can be slower than the load transitions they are meant to follow. Published analysis of AI liquid-cooling deployments describes the resulting gap as thermal debt - heat accumulated between the moment the load steps and the moment the cooling loop catches up - and reports windows of roughly one to two minutes where control is feedback-only, narrowing sharply when the loop is given direct load telemetry in advance.
Air-cooled and hybrid racks are not exempt. In a chassis, the first cooling response to a power step is the fan tray, and the delay between command and delivered airflow is set by the drive's acceleration behaviour, the chassis impedance and the installed system curve. If the command arrives in a fraction of a second but the fan reaches its new operating point several seconds later, that difference is a real thermal debt inside the enclosure - invisible to a controller that reads only its own output.
- Load transition - how rack or tray power moves from baseline to representative peak, and over what time window.
- Control command - when the BMC or BMS issues the new fan, damper or pump setpoint.
- Physical response - when actual RPM, airflow, pressure or flow reach the new operating point.
- Thermal consequence - how far inlet and component temperatures exceed the target before settling.
Record One Timeline Across the Thermal System
A useful test aligns IT telemetry with the cooling equipment response.
Begin at a documented baseline, apply a representative workload transition, hold it long enough for the system to settle, then return to baseline. The required timing and limits come from the actual platform, not a generic number. Synchronize clocks before comparing data; otherwise a controller can appear to react before the condition it was meant to correct.
ASHRAE Guideline 11, Field Testing of HVAC Controls Components, is a useful structural reference even at equipment level. Its sequence runs from basic device testing through system component testing, verification of the sequence of operation, and a defined problem-solving or critical-testing stage, with results captured on documented test forms. Commissioning is a process with defined stages rather than a single measurement, and the same discipline applies to one rack as to a whole plant.
For an AI rack the practical output is one table on one clock: each row is a signal, each cell says what to record and why it matters, and every column shares the same time base.
| Signal | Record | Why it matters |
|---|---|---|
| IT load | Workload state and measured rack or tray power | Defines the thermal input and timestamps the step |
| Server air side | PWM command, actual RPM, chassis differential pressure, inlet and component temperatures | Shows whether chassis fans respond and whether the requested air actually moves |
| Rack air side | Top, middle and bottom inlet temperature with representative pressure | Finds distribution problems, bypass and recirculation |
| Liquid side where fitted | Supply and return temperature, flow and pressure | Shows whether cold-plate or CDU duty is maintained through the transition |
| Controls | BMC and BMS alarm and command timestamps | Separates reported state from physical response |
| Clock source | Polling interval and time source for each system | Sets the resolution of every conclusion drawn later |
Instrument at the Resolution the Event Requires
A fifteen-minute trend log cannot resolve a twenty-second transient.
Sampling rate decides what can be proven. If the BMC polls at one-minute intervals, the fastest events in the transition are simply not in the data; post-processing can only reason about what was captured. Enable the fastest polling the platform supports for the duration of the test, and record the interval itself in the commissioning file so the resolution behind every conclusion is explicit.
Clock discipline matters as much as rate. Align IT, BMC, BMS and any portable instrument to a common time source before the test. A one-second offset between two systems can invert cause and effect, making a fan response appear to precede the load step that triggered it.
Separate the command channel from the physical channel wherever the platform allows it. A logged PWM duty cycle is a request. A tachometer or rotation-detect output is evidence that the rotor is turning. A differential pressure or airflow reading is evidence that the requested air is actually moving through the restriction. All three are needed to describe what happened; any one alone can mislead.
- IT side: rack or tray power, plus a workload marker that timestamps the step.
- Fan side: PWM command, actual RPM, and current draw where the supply allows it.
- Air path: chassis or rack differential pressure with top, middle and bottom inlet temperatures.
- Liquid path where fitted: supply and return temperature, flow and pressure.
- Controls: alarm and command timestamps, each labelled with its own time source.
Run the Step, Not Just the Plateau
The sequence is short; the discipline is in the hold, the repeat and the return.
A defensible power-step test has four phases. Establish and document a baseline at a defined low load. Apply the step to a representative sustained load, using a real workload, a load bank or both. Hold the peak until every monitored thermal node has reached a new steady state, not merely until the room temperature looks acceptable. Then return to baseline and observe the recovery, because a system that cannot shed heat quickly will also struggle on the way up.
Acceptance criteria belong in the commissioning script before the test runs: the maximum excursion above the target envelope, the maximum time to return inside it, and the allowed overshoot during control response. Public electrical acceptance practice already applies this discipline to the power path, where step-load procedures specify voltage and frequency recovery windows rather than a single end point. The thermal path deserves the same structure.
Repeat the step more than once. A single transition can be a coincidence of initial conditions; a repeatable result across several steps, started from different points and separated by a return to baseline, is evidence that the control loop is stable rather than merely fortunate.
| Phase | What is applied | What the record must answer |
|---|---|---|
| Baseline | Documented low-load state held until every temperature is stable | What are the starting temperatures, RPM and pressures, and are they repeatable? |
| Step | Representative transition to a sustained peak load | When did load, command and physical response each move, and in what order? |
| Hold | Peak held until every monitored thermal node stops moving | What was the maximum excursion above target, and how long did settling take? |
| Return | Load removed back to the documented baseline | How fast did the system recover, and did it overshoot on the way down? |
Interactive: What Must Be Proven During a Power Step
Select each stage to see the evidence that belongs in the commissioning record.
Use the normal, one-module-out and replacement states as a compact model of why command, fault response and thermal evidence must agree. It is illustrative, not a substitute for a site-specific commissioning script.
The three states recur at every level of the record. The normal loaded case proves the design, the degraded case proves the margin, and the replacement case proves that maintenance can restore both. A commissioning programme that covers only the first case leaves the two that matter most in an unplanned event unverified.
Read the Air Side First, Because It Moves Fastest
Chassis airflow responds to a load change on a much shorter timescale than facility cooling.
In an air-cooled or hybrid AI rack, the first cooling response to a power step happens at the fan tray, not at the room. That makes the air side the fastest and least forgiving place to look for thermal debt. Read three signals together: the PWM command issued by the controller, the rotor speed reported by the tachometer, and the pressure or inlet-temperature change that should follow from it. When all three agree, the cooling response is real. When the command moves and the temperature does not, the problem sits upstream of the airflow.
Fan behaviour in this window is decided by details that are easy to leave out of a specification: how the drive interprets a changing duty cycle, whether acceleration is limited by design, whether a tachometer or rotation-detect output is actually being monitored, and how the fan behaves where command and supply voltage interact. A fan that performs perfectly at a fixed operating point can still be the wrong part for a system whose load steps in seconds rather than minutes.
MEGA Tech's scope in this layer is the equipment cooling fan itself, not the facility control system. Custom DC fan samples can be specified around frame and mounting, pressure and flow duty, PWM and FG or RD signalling, voltage, connector, noise and environmental requirements, and each candidate is validated in the representative chassis and control path before controlled production. The objective is that the transient behaviour of the delivered part matches the sample that was actually tested.
Test the Degraded States Before Handover
Design margin is meaningful only when a real restriction or failure occurs.
Define approved degraded tests with the equipment and site owners: one fan module unavailable, a filter at its maintenance limit, an allowed service-door condition, loss of one redundant cooling component, or a defined control-sensor fault. Each case needs a stated expected outcome before it is run; otherwise the result is an observation rather than a test.
ASHRAE guidance for liquid-cooled datacom equipment makes the same point at facility level, treating commissioning at full flow to prove hydraulic capacity as a requirement and covering partial-failure and isolation scenarios rather than only the intact design case. That principle scales down to a single fan tray: prove the intact case, then prove the allowed failed case.
- One fan module stopped - confirm alarm latency, the command received by the surviving modules, and the resulting inlet-temperature excursion.
- Filter or restriction at the maintenance limit - confirm the system still holds its thermal target at the defined pressure drop.
- Sensor or signal fault - confirm the fallback behaviour and that it is distinguishable from a genuine thermal event.
- Return to service - confirm the replacement restores the original thermal result and that fault flags clear.
Recommended Fan Starting Points
These catalogue series may be suitable starting points for this application. Final selection depends on the installed airflow, static pressure, voltage, noise target and operating environment.
Request a sample for validation on your equipment.
Engineer FAQ: AI Cooling Commissioning
Questions to settle before releasing an AI rack cooling design.
Q: Is a steady-state thermal test enough? A: No. It does not show response lag, control stability or the margin during a workload transition. A rack can pass a plateau test and still exceed its inlet limit during the step.
Q: Does a BMC fan command prove airflow? A: No. Record actual RPM and correlate it with inlet, component and pressure or airflow evidence. Command and response are different data.
Q: What sampling rate is needed? A: As fast as the platform supports for the duration of the test, and always record what that rate was. A result cannot be more precise than the data behind it.
Q: How long should the peak be held? A: Until every monitored thermal node has stopped moving. The slowest node sets the hold time, not the room temperature.
Q: Why test the return to baseline as well? A: Shed-down behaviour exposes control instability and thermal-mass effects that the ramp up alone can mask.
Q: How many repeats are enough? A: Enough to show the result is repeatable from different starting points, with a return to baseline between steps. One successful transition is an anecdote.
Q: What should a custom fan sample prove? A: Mechanical fit, electrical interface, PWM and FG or RD behaviour, installed pressure and flow duty, thermal acceptance points, and the relevant degraded-state response.
Q: Where does MEGA Tech's responsibility stop? A: At the equipment cooling fan and its validated behaviour in the representative chassis. Facility control architecture, commissioning limits and safety rules remain with the platform and site owners.
Questions for a Design Review
Bring these to the commissioning kickoff, not to the handover meeting.
- Which workload states define the baseline and the step, and who signs them off?
- Which thermal node is slowest to settle, and how long does it take?
- What are the maximum permitted excursion and the maximum recovery time, and where are they written down?
- Which degraded states are approved for test, and which are prohibited?
- Are the IT, BMC and BMS clocks synchronized to a common source, and at what polling interval?
- What evidence will be required to prove that the delivered fan behaves like the validated sample?
