Edge AI: Cloudflare Puts NVIDIA GPUs and TensorFlow on Workers
Cloudflare's edge network, spanning over 200 cities, has long been home to the company's own machine learning workloads for bot detection, anomaly identification, and customer support. Internally, that meant running a GPU cluster for model training and inference. Seven years ago, the edge began supporting custom Lua code for enterprise customers, which eventually evolved into Cloudflare Workers — a platform that lets any developer deploy code in their language of choice across the network.
Adding storage via Workers KV addressed the "algorithms plus data" requirement for programs. Subsequent platform extensions like Workers Unbound, Durable Objects, and Jurisdictional Restrictions rounded out the compute and state story. But modern applications increasingly demand machine learning and deep learning capabilities that need to scale easily, keep models secure, and work with familiar tools.
To close that gap, Cloudflare is extending Workers to support NVIDIA GPUs and TensorFlow. This allows developers to build AI-based applications that run across the Cloudflare network, using either pre-built models for common tasks or custom models for inference.
Why NVIDIA GPUs at the Edge
Before settling on the current approach, Cloudflare evaluated specialized AI accelerator chips and clever model encodings designed to run efficiently on CPUs. The eventual choice was NVIDIA GPUs, driven by their strong support across AI toolkits and the company's existing server footprint in more than 200 cities globally.
The partnership with NVIDIA puts machine learning within milliseconds of the world's online population. This contrasts with prior approaches where models were confined to expensive centralized servers or cloud services locked to specific geographic regions. Low-latency AI inference becomes feasible for anyone deploying on Cloudflare's network.
Keeping models in Cloudflare's data centers also provides a security benefit: developers can deploy custom inference models without shipping them to end-user devices, where they could be extracted. And because models are spread across the network, scaling is handled by the underlying infrastructure.
TensorFlow as the Standard
TensorFlow has become a de facto standard for building and running AI models. Cloudflare's edge AI will leverage TensorFlow directly, so developers can train and optimize models with familiar workflows before deploying them to the edge. In addition to custom models built by users, Cloudflare plans to offer pre-trained models for tasks such as image labeling/object recognition and text detection.
Demo: The Pastel de Nata Recognizer
Cloudflare's Lisbon team built nataornot.com as a demonstration of edge-based AI inference. Visitors can upload a photo of food to determine whether it is a Portuguese pastel de nata (an egg custard tart with cinnamon) or something else.
The underlying model was trained on thousands of images of pasteis and other foods using TensorFlow. At runtime, the model runs on a Cloudflare server equipped with an NVIDIA A100 Tensor Core GPU. The model itself is open sourced for those who want to build their own recognizer.
Getting Started with Workers AI
Developers interested in exploring Workers AI can access it via the getting-started guide at developers.cloudflare.com/workers-ai/get-started.
