Time, at Scale

Meta is rolling out Precision Time Protocol (PTP) across its data centers, replacing Network Time Protocol (NTP) as the primary method for synchronizing clocks. The company argues that PTP's nanosecond-level precision, versus NTP's millisecond-level accuracy, is a prerequisite for the distributed systems it is building toward, including the metaverse and large-scale GPU computing.

The need for tighter synchronization is driven by the sheer number of servers involved in a single transaction. Every message, video call, or game session touches multiple machines, sometimes in different facilities. If one clock is off, users see delays or errors. NTP, which has served as the backbone of network timing for decades, is hitting its limits as networks grow denser and applications become more latency-sensitive.

Accuracy vs. Precision

Two properties matter when synchronizing clocks: accuracy (how close a clock is to the true time) and precision (how close clocks are to each other). PTP was introduced in 2002 to improve both in distributed systems. A network computer, referred to as a Stratum, holds the reference time and sends sync messages to client computers. The challenge is that network latency distorts the time by the time it arrives.

Latency stems from several unavoidable factors:

  • The finite speed of signals over wire or fiber.
  • Variable conversion times in transceivers, which can shift with temperature.
  • The quality of switches and network interfaces.
  • The software, driver, and firmware stack that processes packets.

Latency cannot be eliminated, but it can be compensated for—if it can be measured. That requires two assumptions: consistency (latency stays steady over time) and symmetry (latency is identical in both directions between reference and client). Violations of either assumption degrade synchronization quality.

PTP addresses these issues with two mechanisms: hardware timestamping that improves consistency, and transparent clocks that correct for asymmetry. The telecom industry has used PTP for over a decade, particularly as networks migrate to 5G. Hyperscale data centers have been slower to adopt it, but that is now changing.

Sync Messaging: Two Architectures

NTP-based systems are asynchronous. Each node runs its own clock and periodically checks in with others to stay aligned. As the number of nodes grows, so do these check-ins, adding overhead and slowing the network. NTP implementations use either logical clocks, which timestamp events as a sequence of steps, or physical clocks, which schedule tasks against a local time source. In both cases, engineers deliberately inject delay to compensate for latency.

PTP operates on a different model. Instead of nodes iterating toward a consensus, PTP systems reference a single source of truth—similar to how a smartphone syncs its clock automatically when crossing time zones, as opposed to a microwave clock that requires manual adjustment. This lets PTP work more like the smartphone clock: machines stay aligned to one authoritative time source without the overhead of constant pairwise reconciliation.

The tradeoff is that PTP demands more from the network hardware. Off-the-shelf components were not designed to handle PTP at hyperscale.

Building a Source of Truth

In a PTP deployment, a Server Clock disseminates time to nodes across the network. Traditional designs place boundary clocks or transparent clocks between the Server Clock and end devices. Boundary clocks act as intermediaries, syncing with the Server Clock and relaying time downward. Transparent clocks instead measure and correct for propagation delays. Meta's engineers took a different approach: they removed boundary clocks from the topology entirely, allowing each machine to communicate directly with the Server Clock.

That raises a verification problem. Synchronizing to GPS makes data centers accurate in theory—each site trusts the same satellite constellation—but there is no practical way to confirm that two facilities are actually in sync with each other. To establish its own verifiable source of truth, Meta built a Time Appliance consisting of a GNSS receiver paired with a miniaturized atomic clock (MAC), capable of holding accurate time even if GNSS connectivity drops. A companion Time Card, a PCIe card, turns any commodity server into a time appliance. Both have been open-sourced through the Open Compute Project's Time Appliance Project.

What Better Timing Enables

The immediate beneficiaries of PTP are existing services. Messaging across continents already relies on coordinated servers; tighter synchronization reduces the chance of visible delays. Cloud gaming, which is sensitive to lag, stands to improve as graphics workloads become more intensive. Video conferencing and distributed collaboration tools benefit similarly from reduced jitter and better packet timing.

Looking further out, Meta says PTP could enable synchronization of GPUs across data centers, a capability that would unlock new scale for AI workloads that is hard to achieve today. The company also expects PTP to help systems detect and avoid network congestion more effectively.

To encourage broader adoption, Meta has released its PTP-related work as open source—including the Time Appliance, Time Card, client software, and transparent clock software. Vendor networking equipment will need to support PTP natively for the protocol to become the industry standard, and the company is betting that releasing its designs will accelerate that shift.