
We’re introducing Zoomer, Meta’s comprehensive, automated debugging and optimization platform for AI. Zoomer works across all of our training and inference workloads at Meta and provides deep performance insights that enable energy savings, workflow acceleration, and efficiency gains in our AI infrastructure. Zoomer has delivered training time reductions, and significant QPS improvements, making i

Cloudflare suffered a service outage on November 18, 2025. The outage was triggered by a bug in generation logic for a Bot Management feature file causing many Cloudflare services to be affected.

Most people have heard of open-source software. But have you heard about open hardware? And did you know open source can have a positive impact on the environment? On this episode of the Meta Tech Podcast, Pascal Hartig sits down with Dharmesh and Lisa to talk about all things open hardware, and Meta’s biggest announcements […]

Cloudflare Workers' support for automatic tracing is now in open beta! Export traces to any OpenTelemetry-compatible provider for deeper application observability -- no code changes required

Disaggregated Schedule Fabric (DSF) is Meta’s next-generation network fabric technology for AI training networks that addresses the challenges of existing Clos-based networks. We’re sharing the challenges and innovations surrounding DSF and discussing future directions, including the creation of mega clusters through DSF and non-DSF region interconnectivity, as well as the exploration of alternati

Sapling is a scalable, user-friendly, and open-source source control system that powers Meta’s monorepo. As discussed at the GitMerge 2024 conference session on branching, designing and implementing branching workflows for large monorepos is a challenging problem with multiple tradeoffs between scalability and the developer experience. After the conference, we designed, implemented, and open sourc

We’re sharing details on our journey to scale Meta’s Backbone network to support the increasing demands of new and existing AI workloads. We’ve developed new technologies and designs to address our 10x scaling needs and applying some of these same principles to help extend our AI clusters between multiple data centers. Meta’s Backbone network is […]

We’re presenting Design for Sustainability, a set of technical design principles for new designs of IT hardware to reduce emissions and cost through reuse, extending useful life, and optimizing design. At Meta, we’ve been able to significantly reduce the carbon footprint of our data centers by integrating several design strategies such as modularity, reuse, retrofitting, […]

As we focus on our goal of achieving net zero emissions in 2030, we also aim to create a common taxonomy for the entire industry to measure carbon emissions. We’re sharing details on a new methodology we presented at the 2025 OCP regional EMEA summit that leverages AI to improve our understanding of our IT […]

At Open Compute Project Summit (OCP) 2025, we’re sharing details about the direction of next-generation network fabrics for our AI training clusters. We’ve expanded our network hardware portfolio and are contributing new disaggregated network platforms to OCP. We look forward to continued collaboration with OCP to open designs for racks, servers, storage boxes, and motherboards […]

OpenZL is a new open source data compression framework that offers lossless compression for structured data. OpenZL is designed to offer the performance of a format-specific compressor with the easy maintenance of a single executable binary. You can get started with OpenZL today by visiting our Quick Start guide and the OpenZL GitHub repository. Learn more […]

On August 21, 2025, an influx of traffic directed toward clients hosted in AWS us-east-1 caused severe congestion on links between Cloudflare and us-east-1. In this post, we explain what the failure was, why it occurred, and what we’re doing to make sure this doesn’t happen again.

In this post, we explore the ways we’re evolving Meta’s data warehouse to facilitate productivity and security to serve both human users and AI agents. We detail how we’re developing agents that help users making data access requests to get to the data they need, and that help data owners process requests and maintain security. […]

We’re exploring Meta’s Federation Platform, a scalable set of tools for managing compliance-related tasks, along with Privacy Waves, our method for batching these tasks and ensuring accountability. Together, the Federation Platform and Privacy Waves create a structured, effective, and sustainable approach to operationalizing privacy work, enabling Meta to safeguard user data for the billions of […

Build and deploy real-time, decentralized Authenticated Transfer Protocol (ATProto) apps on Cloudflare Workers.

Hardware faults can have a significant impact on AI training and inference. Silent data corruptions (SDCs), undetected data errors caused by hardware, can be particularly harmful for AI systems that rely on accurate data for training as well as providing useful outputs. We are sharing methodologies we deploy at various scales for detecting SDC across […]

Cloudflare Containers are now available in public beta. Deploy simple, global, and programmable containers alongside your Workers.

We are happy to announce the General Availability of Cloudflare Log Explorer, a powerful product designed to bring observability and forensics capabilities directly into your Cloudflare dashboard.

Workers Builds, our CI/CD product for deploying Workers, monitors build issues by analyzing build failure metadata spread across over one million Durable Objects.

In this post, we explore how Instagram has successfully scaled its algorithm to include over 1000 ML models without sacrificing recommendation quality or reliability. We delve into the intricacies of managing such a vast array of models, each with its own performance characteristics and product goals. We share insights and lessons learned along the way—from […]

As Meta has launched new, innovative products leveraging generative AI (GenAI), we need to make sure the underlying infrastructure components evolve along with it. Applying infrastructure knowledge and optimizations have allowed us to adapt to changing product requirements, delivering a better product along the way. Ultimately, our infrastructure systems need to balance our need to […]

Today we are announcing an alpha version of Pyrefly, an open source Python type checker and IDE extension crafted in Rust. Pyrefly is a static type checker that analyzes Python code to ensure type consistency and help you catch errors throughout your codebase before your code runs. It also supports IDE integration and CLI usage […]

Meta and Quansight have improved key libraries in the Python Ecosystem. There is plenty more to do and we invite the community to help with our efforts. We’ll look at two key efforts in Python’s packaging ecosystem to make packages faster and easier to use: 🚀 Unlock performance wins for developers through free-threaded Python – […]

On April 16, Spotify experienced an outage that affected users worldwide. Here is what happened and what we are going to do about it.

A massive power outage struck significant portions of Portugal and Spain at 10:34 UTC on April 28, disrupting everyday activities and services. The power outage also caused Internet outages across both countries, impacting Internet traffic and connectivity.


Comparing GitHub-hosted vs self-hosted runners for your CI/CD workflows? This deep dive explores important factors to consider when making this critical infrastructure decision for your development team.

Securely store, manage and deploy account level secrets to Cloudflare Workers through Cloudflare Secrets Store, available in beta – with role-based access control, audit logging and Wrangler support.

We’ve improved Observability for Workers by announcing the General Availability of Workers Logs and the introduction of the Query Builder to help you investigate log events across all of your Workers.

You can now add a Deploy to Cloudflare button to your repository’s README when building a Workers application, making it simple for other developers to set up and deploy your project!

The Vercel Marketplace adds new native integrations from Sentry, Checkly, and Dash0. Use the tools you already trust to monitor, measure, and debug your apps with integrated billing, single sign-on, and access to provider dashboards from Vercel.

Deploying like an idiot comes down to a mismatch between the tools you use to deploy and the reward in complexity reduced versus complexity added.

In this article, Edoardo Dusi shares the engineering and architectural choices made by the team at Storyblok and how real-world migration challenges were addressed using modern PHP practices.

Mobile GraphQL is a framework used at Meta for fetching data in mobile applications using GraphQL, a strongly-typed, declarative query language. At Meta it handles data fetching for apps like Facebook and Instagram. Sabrina, a software engineer on Meta’s Mobile GraphQL Platform Team, joins Pascal Hartig on the Meta Tech podcast to discuss the evolution […]

Background im-promp-tu ( im-ˈpräm(p)-(ˌ)tü ) made, done, or formed on or as if on the spur of the moment: improvised composed or uttered without previous preparation: extemporaneous Merriam-Webster On March 18th, 2025, I thought I would look into self-hosted project management solutions — something kanban-y, but.. better? This one does not spark joy. After discovering that Teamhood was awesome (an

On March 21, 2025, multiple Cloudflare services, including R2 object storage experienced an elevated rate of error responses. Here’s what caused the incident, the impact, and how we are making sure it doesn’t happen again.

Cloudflare Aegis provides dedicated egress IP addresses for Zero Trust origin access strategies, now supporting BYOIP and customer-facing configurability, with observability of Aegis IP utilization coming soon.

By building and integrating a new heuristics framework into the Cloudflare Ruleset Engine, we now have a more flexible system to write rules, deploy new releases rapidly, and give Bot Management customers the explainability and control they were asking for.
CL
Curtis Lowder, Brian Mitchell·March 19, 2025SRE & Ops 
A look into building IssueOps workflows on GitHub to do everything from CI/CD to handling approvals and more.

The growth of data and need for increased power efficiency are leading to innovative storage solutions. HDDs have been growing in density, but not performance, and TLC flash remains at a price point that is restrictive for scaling. QLC technology addresses these challenges by forming a middle tier between HDDs and TLC SSDs. QLC […]

On Thursday, February 6, 2025, we experienced an outage with our object storage service (R2) and products that rely on it. Here's what happened and what we're doing to fix this going forward.
SJ
Silverlock, Javier·February 7, 2025SRE & Ops 
We’re sharing details about Glean, Meta’s open source system for collecting, deriving, and working with facts about source code. In this blog post we’ll talk about why a system like Glean is important, explain the rationale for Glean’s design, and run through some of the ways we’re using Glean to supercharge our developer tooling at […]

On November 14, 2024, Cloudflare experienced a Cloudflare Logs outage, impacting the majority of customers using these products. During the ~3.5 hours that these services were impacted, about 55% of the logs we normally send to customers were not sent and were lost. The details of what went wrong and why are interesting both for customers and practitioners.
JT
Jamie, Tom Walwyn·November 26, 2024SRE & Ops 
AI plays a fundamental role in creating valuable connections between people and advertisers within Meta’s family of apps. Meta’s ad recommendation engine, powered by deep learning recommendation models (DLRMs), has been instrumental in delivering personalized ads to people. Key to this success was incorporating thousands of human-engineered signals or features in the DLRM-based recommendation syst

Incident Management takes time Incidents need responders that are trained and experienced. At Slack, training is a foundation of our incident management program. Self-service training and live courses based mainly on prepared content are one piece of the puzzle, but there can be a missing piece in many organizations. How can staff get practical experience…
SN
Scott Nelson Windels·November 14, 2024SRE & Ops 
Workers Builds, an integrated CI/CD pipeline for the Workers platform, recently launched in open beta. We walk through how we built this product on Cloudflare’s Developer Platform.
SJ
Serena, Jacob Hands·October 31, 2024SRE & Ops 
On October 30, 2024, cloud hosting provider OVHcloud (AS16276) suffered a brief but significant outage. Within this post, we review Cloudflare’s perspective on this outage.
BD
Bryton, David Belson·October 30, 2024SRE & Ops 
Cloudflare Workflows is now in open beta! Workflows allows you to build reliable, repeatable, long-lived multi-step applications that can automatically retry, persist state, and scale out. Read on to learn how Workflows works, how we built it on top of Durable Objects, and how you can deploy your first Workflows application.

Access for Infrastructure, BastionZero’s integration into Cloudflare One, will enable organizations to apply Zero Trust controls to their servers, databases, Kubernetes clusters, and more. Today we’re announcing short-lived SSH access as the first available feature of this integration.
GA
Goldbe, Ann Ming Samborski·October 23, 2024SRE & Ops 
At Open Compute Project Summit (OCP) 2024, we’re sharing details about our next-generation network fabric for our AI training clusters. We’ve expanded our network hardware portfolio and are contributing two new disaggregated network fabrics and a new NIC to OCP. We look forward to continued collaboration with OCP to open designs for racks, servers, storage […]

At the Open Compute Project (OCP) Global Summit 2024, we’re showcasing our latest open AI hardware designs with the OCP community. These innovations include a new AI platform, cutting-edge open rack designs, and advanced network fabrics and components. By sharing our designs, we hope to inspire collaboration and foster innovation. If you’re passionate about building […]

This roundup blog post shares the latest new features and capabilities at Cloudflare. Learn more about new Digital Experience (DEX) monitoring notifications and seamless access to Cloudflare Gateway with China Express.
GN
Guy Nir, Dong Zhang·October 9, 2024SRE & Ops 
The Kubernetes team runs several multi-tenant clusters across Cloudflare’s core data centers. When multi-tenant cluster isolation is too limiting for an application, we use KubeVirt. KubeVirt is a cloud-native solution that enables our developers to run virtual machines alongside containers.

On September 17, 2024, during planned routine maintenance, Cloudflare stopped announcing 15 IPv4 prefixes, affecting some Business plan websites for approximately one hour. During this time, IPv4 traffic for these customers would not have reached Cloudflare and users attempting to connect to websites using addresses within those prefixes would have received errors.

At Slack, we manage tens of thousands of EC2 instances that host a variety of services, including our Vitess databases, Kubernetes workers, and various components of the Slack application. The majority of these instances run on some version of Ubuntu, while a portion operates on Amazon Linux. With such a vast infrastructure, the critical question…
AGArchie Gunasekara·September 17, 2024SRE & Ops 
We’re sharing more about the role that reinforcement learning plays in helping us optimize our data centers’ environmental controls. Our reinforcement learning-based approach has helped us reduce energy consumption and water usage across various weather conditions in our data centers. Meta is revamping its new data center design to optimize for artificial intelligence and the […]

[…]


AWS CodeCommit is discontinuing new customer access and will no longer introduce new features. Learn how to migrate to GitHub Enterprise and why it’s the best option for you.
MM
Melody Mileski, Jun Pak·August 27, 2024SRE & Ops 
We are introducing a new metric— real-time server fleet utilization effectiveness —as part of the RETINAS initiative to help reduce emissions and achieve net zero emissions across our value chain in 2030. This new metric allows us to measure server resource usage (e.g., compute, storage) and efficiency in our large-scale data center server fleet in […]