ASICs at the Edge

The Chips That Run the Network Edge

Cloudflare’s network spans more than 200 cities across over 100 countries, and moving traffic through that infrastructure depends on a handful of foundational technologies. One of the most important is the ASIC.

What an ASIC Is — and Why Networks Rely on Them

An Application Specific Integrated Circuit is, as the name suggests, a chip built for one narrowly defined job. That makes it fundamentally different from a general-purpose CPU, or even a GPU, which has found broad use in parallel workloads like video encoding and computer vision through technologies such as CUDA and OpenCL.

ASICs trade flexibility for efficiency. Because they only contain the hardware needed for their specific function, they are smaller and require less power and cooling than more general hardware. That translates to more compact network devices and lower data-center power draw. Manufacturing simpler chips also tends to be more reliable and cheaper.

The trade-off is that features have to be embedded in hardware at design time. If a new protocol or specification appears after a chip has been fabricated, existing silicon simply cannot support it — VXLAN is a good example of a technology that required new hardware support. For networking, that constraint is usually acceptable. The industry evolves slowly, with technologies like IPv6 and MPLS taking years or decades to reach broad deployment. Chips can therefore be developed on a longer cycle with larger generational jumps; Broadcom, for instance, took two years to move from Tomahawk 3 to Tomahawk 4, but doubled throughput in the process.

From FPGA Prototype to Fixed Silicon

Developing an ASIC is a long process, and errors are costly: a hardware defect means the entire design must be redone. Engineers first prototype in an FPGA (Field Programmable Gate Array), programming the intended functionality in a Hardware Description Language like Verilog to verify compatibility. Only after that stage is complete does the design go to a foundry, where the packet-processing pipeline is baked into the chip. At that point, unlike an FPGA, it can no longer be reprogrammed.

The market for ASICs is also narrow — few companies buy them in bulk to build networking equipment — which pushes unit costs up. Iteration cycles are accordingly slow, typically incremental: port speeds advance in steps (1G → 10G → 25G → 40G → 100G → 400G → 800G), tied to upgrades in the SerDes (Serialiser/Deserialiser) portion of the chip. Supporting new protocols is harder still, and may take several development cycles before reaching silicon.

How ASICs Make Forwarding Decisions

In network equipment, ASICs handle the switching and routing of packets and act as a first line of defense in the form of a stateless firewall. Fast lookups are critical, so ASICs use TCAM (Ternary Content-Addressable Memory) to store tables of forwarding information, ACL (Access Control List) rules, and CoS (Class of Service) priorities.

TCAM is a physical implementation of a key-value store, operating in a fundamentally different way from traditional RAM. In RAM, data is retrieved by address. In CAM and TCAM, you query directly by content, and the memory returns a match — often within a single clock cycle — by referencing the exact binary representation of a word, such as an IP address like 203.0.113.0 (11001011.00000000.01110001.00000000).

TCAM adds a crucial third state beyond the binary 0 and 1: an “ignore” or wildcard state. That allows a single TCAM entry to cover a range of sequential words — in networking terms, an IP prefix. Instead of storing all 254 IPs in a subnet individually, TCAM can represent the whole 203.0.113.0/24 block as one entry. The ASIC can then answer a question like “where do I send packets destined for 203.0.113.19?” in one cycle. The reply is usually a pointer into regular RAM, where details such as output port and firewall requirements are stored.

Routing, Switching, and the Forwarding Plane

Networking has two primary components: switching, which interconnects devices on the same network, and routing, which connects those switched networks into a larger internetwork — eventually the Internet itself. Routers steer traffic between these networks, typically using BGP to exchange reachability information, or NLRI (Network Layer Reachability Information), for IP prefixes.

Because there are many possible paths between two points, routers store all learned routes in the RIB (Routing Information Base). A BGP daemon then runs a decision process to select the ideal path for each destination, and stores that best route in the FIB (Forwarding Information Base). The FIB is a subset of the RIB, holding only the current optimal paths while the RIB retains all alternatives for fast failover.

Several specific functions happen in the ASIC to move packets quickly:

• FIB install: Once a router has calculated its FIB, the ASIC writes it into TCAM so lookups happen at hardware speed.
• Packet forwarding lookups: Each received packet triggers a TCAM lookup to determine the appropriate next hop.
• Stateless firewall: The ASIC checks packet types against TCAM rules, such as dropping traffic sourced from RFC1918 space. This occurs at line rate. Stateful inspection is far more complex; Cloudflare moves that responsibility to its compute nodes, reserving routers for stateless filtering.
• Encapsulation: For overlay networking, the ASIC can encapsulate packets inline in protocols like GRE, avoiding a second software trip through the pipeline and the latency that would add.
• Advanced protocol support: MPLS, EVPN, VXLAN, SDWAN, SDN and similar features increase chip complexity, so not all ASIC vendors implement every feature across their product lines.

The Silicon Behind the Switch

Choosing switching hardware comes down to a few key technical trade-offs. The main differentiators are throughput (bits pushed per second), buffer size (memory available during contention), programmability (how easily third parties like Cloudflare can work directly with the ASIC), and the feature set beyond basic routing and switching. Requirements vary widely: what a hyperscale edge provider needs from its routers looks nothing like what a typical enterprise wants from its ToR switch—and even within a single network, different layers demand different silicon.

Broadcom

Broadcom dominates the merchant silicon market, holding roughly 59% of the Ethernet integrated circuits market. Much of that position came through acquisition, most notably Dune Networks, whose technology became the StrataDNX line. Historically, working with Broadcom parts meant navigating heavy NDAs to access their SDK. That changed when Broadcom open-sourced the SDK, easing programmability for outside developers.

Broadcom's portfolio spans three distinct product families:

• Tomahawk targets the enterprise space with cheap, high-speed, low-latency switching. Tomahawk 4 reaches 25.6Tbps in 7nm, but trades away advanced features like VXLAN and carries essentially no buffer. Juniper's QFX5200 uses this silicon.
• StrataDNX (Arad, QumranMX, Jericho) came from the Dune Networks acquisition and offers deep buffers at high bandwidth. Jericho2 is a 10Tbps deep-buffer chip. Arista's DCS-7280SR, used at some Cloudflare edge sites as an edge router, runs on Jericho. Cisco bases its NCS5500 routers on QumranMX, and fabric chips can link these into 48x400G port switches.
• Trident adds a fuller feature set to Tomahawk-class speeds. Trident4 reaches 12.8Tbps with low latency, but like Tomahawk lacks meaningful buffer capacity—limiting its use at Cloudflare, where buffer is needed to handle switching across different port speeds. Arista's 7050X and 7300X are Trident-based.

Intel and the Programmable Tofino

Intel is known for stable 10G NICs, not ASICs. An early attempt via the Fulcrum acquisition and FM6000 series never gained traction. The 2019 Barefoot acquisition changed that. Barefoot's Tofino is built on a PISA (Protocol Independent Switch Architecture) and is programmable via P4, a language for defining data-plane forwarding behavior. This breaks from the traditional model where the ASIC data plane is fixed and not directly programmable. A further advantage: P4 programs can undergo formal verification to confirm forwarding behavior. (OpenFlow attempted this kind of shift but never gained momentum.)

The top-end Tofino 1 has 6.5Tbps linerate capacity, and because the chip is programmable, its feature set is as extensive as you make it. Buffer memory is limited, so these are not yet deployable as edge devices. Both Arista (7170 series) and Cisco (Nexus 34180YC and 3464C) ship Tofino-based hardware.

Mellanox/Nvidia

Mellanox—now owned by Nvidia—supplies the 25G NICs used in Cloudflare compute nodes and also builds ASICs, mostly for switching. The current Spectrum 3 provides 12.8Tbps with an extensive feature set including Deep Packet Inspection and NAT, enabling dense high-speed platforms up to 25.6Tbps. Buffering is minimal at 64MB. Mellanox ships its own hardware but, unlike some competitors, does not lock it to its own OS; options include Cumulus Linux (also Nvidia-acquired). While Cloudflare relies on Mellanox NICs, no Mellanox switching silicon currently runs in its network.

Juniper's In-House Families

Juniper is Cloudflare's largest network equipment supplier. It buys Broadcom silicon for some products but also derives from two in-house families: Express for switching-heavy tasks and Trio for feature-intensive routing.

Express and Triton

The Express family sits in the same application space as StrataDNX: high bandwidth plus a broad feature set. The Q5 (Paradise) generation handles 500Gbps per chip. Combined with a fabric chip in a Clos network architecture, line cards reach 12Tbps. The Q5 boasts 4GB of buffer per ASIC housed in external Hybrid Memory Cube (HMC) modules, which also store the FIB and MAC tables (no TCAM is used). The QFX1000 switches (QFX10002-36Q, QFX10002-60C, QFX10002-72Q, and QFX10008) run on Q5 and are deployed at Cloudflare as edge routers and core aggregation switches.

ExpressPlus (ZX) doubles per-chip bandwidth to 1Tbps and, with an integrated Clos fabric, reaches 6Tbps in a 2U PTX10002 form factor. It also scales up logical capacity with larger buffers, FIB storage, and ACL space. ExpressPlus drives part of the PTX router line alongside the newest family member, Triton (BT), which reaches 3.6Tbps per chip. Both Triton and ExpressPlus support 400GE.

Trio for Routing

The Trio family serves the MX routing platform and is in its fifth generation. Trio Eagle (4.0, EA) appears on MPC7E line cards with 400Gbps forwarding and substantial buffer and TCAM resources. Trio Penta (5.0, ZT) is the current generation, rating 500Gbps per ASIC, which scales to 4Tbps per line card, and adds hardware offload for features such as MACSec and Layer 3 IPsec. The MPC10E and MPC11E line cards (installable in MX chassis including the MX480) carry Penta.

Cisco's Custom Chips for Data Center and ToR

Cisco remains the largest network vendor, mixing merchant silicon and homegrown ASICs across its fleet. Cloudflare previously ran Cisco ASR 9000s as edge routers; today Cisco is used heavily for Top-of-Rack switching with the Nexus 5000 and 9000 series.

Bigsur is custom silicon inside the Nexus 6000 line—specifically the Nexus 5672UP and 6001 models. In the Nexus 5672UP, seven Bigsur ASICs interconnect to provide 10G and 40G connectivity. Cisco is far less forthcoming about its ASIC specifications, but for Cloudflare's use these are simple Layer 2 forwarding switches using Virtual Output Queueing. The Bigsur doesn't offer the TCAM or buffer capacity of Juniper's silicon.

The Tahoe ASIC (also called the Leaf Spine Engine, or LSE), found in the Nexus 9300-EX switches, is a maturation of Bigsur. It offers 1.6Tbps, denser port configurations, support for advanced VXLAN and EVPN fabrics, port flexibility for 10G through 100G, and increased 40MB buffers. Tahoe is deployed extensively across both Cloudflare's edge and core data centers.

The ASIC Landscape at the Edge

The hardware that powers network edges is getting more specialized, and the decision of which ASIC to build around is now a core architectural choice rather than a supply-chain detail. From hyperscale cloud operators down to enterprise campuses, the trade-offs between fixed-function silicon, programmable data planes, and merchant silicon versus custom designs are reshaping what routers and switches can do.

The Merchant Silicon Mainstream

Broadcom remains the dominant force in merchant switching silicon. Its StrataXGS family spans access to data center tiers, while the Tomahawk series targets high-density, high-bandwidth leaf and spine deployments. The Tomahawk 4 pushes 25.6 Tbps per device, making it a common choice for the highest-throughput fixed-form-factor switches. For more feature-rich routing and data center gateway roles, the StrataDNX family, such as the BCM88690, builds in deeper buffers and richer feature sets at the cost of raw port count.

This split reflects a fundamental tension: pure speed versus functional depth. High-bandwidth parts like Tomahawk often sacrifice buffering and complex packet processing, pushing those responsibilities up the stack or into adjacent hardware.

Programmable Data Planes and P4

The rise of programmable ASICs has altered the calculus for teams that need custom behavior without custom silicon. Intel, after acquiring Fulcrum Microsystems, markets its FM5000 and FM6000 Ethernet switch series, which offer some programmability but are fixed in their forwarding pipelines. The real shift came with the P4 language and the hardware designed for it, notably the Barefoot Tofino family.

Tofino-based switches, such as those from Arista’s 7170 line and Cisco’s Nexus 9000 series with Tofino inside, allow operators to define their own parsing, match-action tables, and packet generation logic. This is not microcode tweaking; it is a fundamentally redefinable pipeline. Use cases include telemetry at line rate, in-network load balancing, and precise congestion control that goes beyond what fixed-function chips can express. The P4 model treats the switch as a programmable pipeline where the control plane lives outside the chip, addressing the need for protocol innovation without waiting years for new silicon revisions.

The Hyper-Scale Advantage: Bespoke Chips

At the very top of the market, a different strategy emerges. Companies like Google and SpaceX are not just buying ASICs; they are designing them. Google’s TPU project shows the pattern of vertical integration, and the ecosystem for that work is visible in job listings for FPGA and ASIC design engineers, even at aerospace companies like SpaceX, which needs custom processing in radiation-tolerant, power-constrained environments.

Broadcom defends its merchant position with scale: the wired infrastructure segment alone is a major revenue moat, and the same chip designs now feed Apple’s smartphones after the company landed supplier deals for wireless components. That scale funds the relentless march to 51.2 Tbps-class devices and beyond, keeping merchant parts ahead of most operators’ needs.

What the Edge Actually Needs

Comparing the priorities of edge networks is an exercise in trade-offs. For a campus core, buffer depth and 400G port economics matter more than exotic programmability. The Cisco Nexus 9300-EX platform documentation is instructive here: feature breadth, from VXLAN to advanced QoS, is what drives value in that segment. Conversely, for massively parallel data center fabrics, the Tomahawk line’s lack of deep buffering is acceptable because the fabric redistributes load dynamically across tens of thousands of ports.

With 25.6 Tbps of switching now a shipping reality, attention shifts to the next bottleneck: systems that can actually feed those ports. A UT 48-port 400G disaggregated core router layout illustrates one route: eliminate dedicated fabric chips and rely on a single, high-radix ASIC to connect every port directly. This "distributed core" model works only with careful traffic engineering, but it avoids the power and latency cost of stitching multiple smaller devices together.

Practical footnotes from various slidesets and whitepapers show that disaggregation is not just a data center fad. The Juniper ACX and MX lines have leaned into this trend by offering white-box and bare-metal versions, some running on Broadcom silicon and some on Juniper’s own Express Plus parts.

Conclusions on the Edge

Choosing the next edge network platform is rarely about identifying the single best ASIC. The interconnected decisions involve:

• Software ecosystem maturity and API openness—will your team be able to automate around the chip, or are you locked into a vendor’s CLI?
• Interoperability risk with existing routing stacks and orchestration layers when placing a new data plane beneath them.
• The pace of port-race economics; the merchant chip shipped last year is half the bandwidth of the one shipping now, and that sequence is not slowing.

The engineers behind these decisions stand on the detailed public work of network architects who have documented the new Juniper MX boards in Day One books, mapped the QFX10000 series internals, and maintained living summaries of switching ASICs. The collective output of that community doesn't settle the debate between "speed (Tomahawk) vs. depth (DNX)" vs. "complete reprogrammability (Tofino)"; it frames it precisely, with numbers down to the per-slice buffer bytes.

The pace of leading-edge merchant switching reminds us that buying last year’s fixed-config box to save money is often a false economy—the cap-ex gap closes quickly once you cost in operational overhead and the lost ability to turn network flexibility into competitive innovations.