Simulator-based reinforcement learning for data center cooling optimization

Reinforcement learning trims cooling overhead in Meta data centers

Cooling is the largest consumer of energy and water in a data center after the IT load itself. Meta has been working on reducing that overhead since 2021, using a simulator-based reinforcement learning (RL) approach to control supply airflow. In a pilot at one data center region, the technique cut supply fan energy consumption by an average of 20% and water usage by 4% across a range of weather conditions.

Most of Meta’s existing facilities use 100% outside air with evaporative cooling, maintaining temperatures between 65°F and 85°F and relative humidity between 13% and 80%. Because conditioning that air consumes both energy and water, minimizing the volume of supply airflow that must be treated is a key efficiency lever. Meta says the same methodology it has piloted will apply to its next-generation data center designs optimized for AI workloads.

How the cooling system is controlled

Meta’s two-tiered penthouse design draws outside air through modulating dampers into a mixing room, where it can be blended with warmer server exhaust if the outdoor air is too cold. The air then passes through filters and a misting chamber for evaporative cooling and humidification before a fan wall pushes it down through floor openings into the server hall. Hot aisle air is exhausted back outside with the help of relief fans.

Water is consumed in two operations: evaporative cooling, which lowers temperature when outside air is too warm, and humidification, which adds moisture when air is too dry. A building management system (BMS) monitors and adjusts three main control loops — temperature, humidity, and airflow setpoints — to keep conditions inside the operating envelope.

The airflow setpoint is normally derived from a handful of inputs such as IT load, cold aisle temperature, and the differential pressure between cold and hot aisles. The relationship between these values is difficult to model accurately because they are coupled across locations in the facility and depend on complex local boundary conditions. Yet it is the airflow setpoint that most directly drives supply fan energy and water consumption, since temperature and humidity limits are fixed. That makes it the natural target for optimization.

Training a policy offline before touching real controls

Meta frames data center cooling control as a sequential decision problem. The state is the collection of sensor readings for outside air, supply air, cold aisle, hot aisle, and IT load. The action is adjusting the airflow setpoint. The policy maps the state to the appropriate action, and the goal is to optimize that policy for energy and water efficiency.

The team chose an offline, simulator-based RL approach rather than letting an online model directly drive live controls. An online policy carries risk of thermal unsafety or violations of service requirements. Instead, the RL agent starts from real historical observations and explores potential actions in the simulator, which predicts the resulting state and reward. The agent collects the state-action pairs with the best outcomes and uses them to update the policy model, parameterized by a deep neural network.

The simulator is physics-based, taking time series inputs such as weather, IT load, and setpoint schedules. It is parameterized with building characteristics including geometry, construction materials, HVAC configuration, component efficiencies, and control strategies. Differential equations model the dynamic response — thermal load, energy use, cold aisle temperature, and differential pressure profiles.

Using a physics-based simulator rather than purely historical data means the RL model can be trained against scenarios that have not yet occurred. That improves reliability during outlier events and enables faster deployment in newly commissioned facilities. The simulator also enforces operational constraints: cold aisle temperature rise must stay below a threshold, and positive pressurization from cold aisle to hot aisle must be maintained to prevent recirculation-induced parasitic heat.

Pilot results across weather conditions

Meta began a pilot in 2021 at one data center region, having the RL model directly control the supply airflow setpoint. The new setpoint fluctuates more than the original BMS setpoint — driven by supply air temperature and server load cycles throughout the day — but the facility remained within specification. Both average and maximum cold aisle temperatures stayed below limits even with reduced airflow.

The savings vary by supply air temperature. In hot, dry conditions, when evaporative cooling or humidification is required, the reduced airflow also cuts water consumption. Over the multi-year pilot, those gains averaged 20% for supply fan energy and 4% for water usage.

Applying the approach more broadly

Meta says it is rolling out the RL-based cooling control to existing data centers and expects the savings to accumulate over the next couple of years. The same methodology is also being factored into the design phase of new data centers built for AI, so they can be optimized for sustainability from the start of operations. The company positions this as a move toward automated, continuously tuned environmental controls that reduce manual effort and improve efficiency.