How we used OpenBMC to support AI inference on GPUs around the world

Refreshing the BMC Layer for GPU Workloads

Delivering serverless GPU inference across Cloudflare's global network required more than just adding accelerators to existing hardware. A significant part of the effort involved updating the firmware on our Baseboard Management Controllers (BMCs)—the embedded microprocessors responsible for remote power management, sensors, serial console, and virtual media on most servers.

To manage these controllers efficiently and keep the firmware auditable, Cloudflare relies on OpenBMC, an open-source stack from the Open Compute Project (OCP). The work below highlights the practical steps we took to support new GPUs in our fleet using this firmware.

Thermal Tuning Through the Fan PID Controller

When we added new GPUs to existing chassis, power delivery was not the bottleneck—our power supplies had sufficient headroom. The real constraint was cooling. Initially, we took an aggressive approach: set the fans to blast directly at the GPU, assuming it would run at Thermal Design Power (TDP). That was not enough. Under full stress, the GPU exceeded 95˚C. Adding a second fan brought the temperature down to 65˚C, giving us a stable baseline for further work.

With the baseline established, we moved on to fine-tuning the fan PID (Peripheral Integral Derivative) controller. The control output is governed by:

Here, u(t) is the control output, e(t) is the error (target temperature minus current temperature), and Kp, Ki, and Kd are the proportional, integral, and derivative gains. Each component has a specific role:

• Proportional gain – governs the speed at which the controller reacts to the error.
• Integral gain – accumulates error over time to eliminate steady-state error (where the controller settles, but not at the target setpoint).
• Derivative gain – responds to the rate of change, helping dampen oscillations.

Our tuning targets were: avoid fan speed oscillations (which cause mechanical wear over a five-year server lifecycle), approach the setpoint quickly, and eliminate steady-state error. Iterating on these constants is tedious, but OpenBMC makes it manageable. The fan configuration is stored in JSON files, and the PID application generates CSV logs, so we can tune and visualize changes quickly.

Our initial attempt, with fans at full load, showed difficulty locking onto the setpoint. Adjusting the integral and derivative gains reduced the oscillations, though the controller initially wanted to settle around 70˚C instead of our 65˚C target—a classic steady-state error. Increasing the proportional gain sped up the approach and removed that offset. After several iterations, we had a response with minor initial overshoot, which was acceptable for production.

Establishing GPU-to-BMC Communication

The first step to tuning the fans was getting accurate temperature data from the GPU itself. That required a path from the BMC to the PCIe slot. Reviewing the motherboard schematics, we found an SMBus (System Management Bus) line running from the BMC through a mux to the PCIe slot—similar to I²C but with different electrical and clock requirements.

With the physical path mapped, we turned to software. OpenBMC’s Linux-based environment allows us to bind drivers and run tools easily. For inventory data, we used the eerpog tool to issue block reads over SMBus to the GPU’s EEPROM:

~$ eeprog -f -16 /dev/i2c-23 0x50 -r 0x00:200
eeprog 0.7.5, a 24Cxx EEPROM reader/writer
Copyright (c) 2003 by Stefano Barbato - All rights reserved.
  Bus: /dev/i2c-23, Address: 0x50, Mode: 16bit
  Reading 200 bytes from 0x0
<redacted> Ver 0.02

For temperature, we worked with the tmp75 sensor—standard on many server components. After binding the driver to the appropriate bus address, we confirmed the sensor was active:

{
    "Exposes": [
        {
            "Address": "0x4F",
            "Bus": "23",
            "Name": "GPU_TEMP",
            "Thresholds": [
                {
                    "Direction": "greater than",
                    "Name": "upper critical",
                    "Severity": 1,
                    "Value": 92
                },
                {
                    "Direction": "less than",
                    "Name": "lower non critical",
                    "Severity": 0,
                    "Value": 30
                }
            ],
            "Type": "TMP75"
        }
    ],
    "Name": "****************",
    "Probe": "xyz.openbmc_project.FruDevice({'BOARD_PRODUCT_NAME': *********})",
    "Type": "NVMe",
    "xyz.openbmc_project.Inventory.Decorator.Asset": {
        "Manufacturer": "$BOARD_MANUFACTURER",
        "Model": "$BOARD_PRODUCT_NAME",
        "PartNumber": "$BOARD_PART_NUMBER",
        "SerialNumber": "$BOARD_SERIAL_NUMBER"
    }
}

With raw sensor access working, we moved to Entity-Manager. This OpenBMC component translates physical hardware into software-visible objects using JSON configurations. It probes the I²C buses for EEPROMs and matches the responses against the Probe member in each configuration. Once matched, the defined sensors and inventory items are exposed via IPMI or Redfish. After configuring the GPU's temperature and FRU data through Entity-Manager, both were visible as GPU_TEMP and the expected FRU inventory over IPMI.

$~ ipmi 517m206 sdr |grep GPU_TEMP
GPU_TEMP         | 39 degrees C      | ok
$~ ipmi 517m206 fru print 151
FRU Device Description : <redacted> (ID 151)
 Board Mfg Date        : Mon Mar 27 18:13:00 2023 UTC
 Board Mfg             : <redacted>
 Board Product         : <redacted>
 Board Serial          : <redacted>
 Board Part Number     : <redacted>

Why Firmware Flexibility Matters

The GPU deployment underscored a core benefit of OpenBMC: firmware can be updated and tuned with the hardware, without being tied to traditional Original Design Manufacturer (ODM) update cycles. We still work closely with our partners, but OpenBMC gives us the ability to adapt server firmware in-house, quickly and transparently.