Workers AI: Turning model inference into a platform primitive
AI has moved from a specialist topic to a mainstream part of application development, but running models still carries a lot of operational baggage. Historically, getting value from machine learning meant mastering model management and standing up GPU infrastructure yourself. Cloudflare is taking a different angle with Workers AI, an inference-as-a-service offering that runs open models on serverless GPUs distributed across the company's network.
The goal is to remove the infrastructure barrier entirely: no GPU procurement, no cluster setup, no model serving stack. Workers AI is designed as a building block for the Cloudflare developer platform, accessed from Workers or Pages, and also exposed as a REST API so it works with any stack. It is pay-as-you-go, runs close to end users, and is private by default.
What ships at launch
The launch catalog is a curated set of popular open source models covering common inference tasks:
- Text generation:
meta/llama-2-7b-chat-int8 - Automatic speech recognition:
openai/whisper - Translation:
meta/m2m100-1.2 - Text classification:
huggingface/distilbert-sst-2-int8 - Image classification:
microsoft/resnet-50 - Embeddings:
baai/bge-base-en-v1.5
Model browsing happens inside the Cloudflare dashboard, with per-model logs and analytics slated to follow. A partnership with Hugging Face will expand the catalog, letting developers run a subset of Hugging Face models directly from Workers AI.
Access without vendor lock-in
Workers AI is intentionally platform-agnostic. Developers inside the Cloudflare ecosystem get a binding, but external users consume the service through a straightforward REST interface.
curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/m2m100-1.2b \
-H "Authorization: Bearer {API_TOKEN}" \
-d '{ "text": "I'll have an order of the moule frites", "target_lang": "french" }'{
"result": {
"answer": "Je vais commander des moules frites"
},
"success": true,
"errors":[],
"messages":[]
}From zero to deployed Llama 2
The developer experience is the core test: can a developer get a model running in minutes without learning infra? A walkthrough with Llama 2 shows the path. With a Cloudflare account, Node, and NPM in place, setup follows these steps.
Create a project
$ npm create cloudflare@latestworkers-ai, choose the Hello World script, opt into TypeScript and Git, and skip deployment for now.
cd workers-aiBind Workers AI to the worker
Add the binding to wrangler.toml. The binding gives the worker access to Workers AI without key management:
[ai]
binding = "AI" #available in your worker via env.AIPages Functions get the same capability — see the Functions Bindings docs for those details.
Install the client library
npm install @cloudflare/aiWrite the inference call
The worker code in source/index.ts calls Llama 2 with a prompt and returns the model output.
import { Ai } from '@cloudflare/ai'
export default {
async fetch(request, env) {
const ai = new Ai(env.AI);
const input = { prompt: "What's the origin of the phrase 'Hello, World'" };
const output = await ai.run('@cf/meta/llama-2-7b-chat-int8', input );
return new Response(JSON.stringify(output));
},
};Test remotely
Local testing requires the --remote flag because models execute exclusively on Cloudflare's GPU network today, and you'll be asked to log in.
$ npx wrangler dev --remote{
"response": "Hello, World is a common phrase used to test the output of a computer program, particularly in the early stages of programming. The phrase "Hello, World!" is often the first program that a beginner learns to write, and it is included in many programming language tutorials and textbooks as a way to introduce basic programming concepts. The origin of the phrase "Hello, World!" as a programming test is unclear, but it is believed to have originated in the 1970s. One of the earliest known references to the phrase is in a 1976 book called "The C Programming Language" by Brian Kernighan and Dennis Ritchie, which is considered one of the most influential books on the development of the C programming language.
}Deploy
$ npx wrangler deploy
# Outputs: https://workers-ai.<YOUR_SUBDOMAIN>.workers.devPrivacy posture and data handling
From Cloudflare's founding pillars — security, reliability, performance — privacy has become an explicit priority on the platform. Workers AI is private by default: no training on your data or conversations, and models do not learn from usage. This applies to the base service rather than being gated behind an enterprise tier. Future data localization support is on the roadmap, tied to a GPU rollout plan that starts with seven sites, expands to around 100 by the end of 2023, and reaches near-global coverage by the end of 2024.
Beyond inference: Vectorize and AI Gateway
Inference alone only goes so far. LLMs have a knowledge cutoff and no business context, so Workers AI sits alongside two complementary services.
Vectorize is a vector database designed to work with Workers AI, enabling retrieval-augmented workflows. A typical pattern looks like this:
- Generate embeddings from your knowledge base using a Workers AI embedding model.
- Store those embeddings in Vectorize to seed the database.
- When a user asks a question, generate an embedding for that query.
- Query Vectorize with that embedding to pull the most relevant context.
- Combine the retrieved text with the original question to build a context-aware prompt.
- Send that prompt through a Workers AI LLM for the final answer.
AI Gateway covers the opposite use case: if you run models with other providers, routing those APIs through the gateway adds caching, rate-limiting, analytics, and logging — useful for hardening endpoints, controlling cost, and data loss prevention.
Beta access and current limits
Workers AI is now available as an open beta for all Workers plans, free or paid. Because this is an early release, Cloudflare cautions against using it for production applications; limits and access terms may still change. Initially, rate limits are set per model — for example, @cf/meta/llama-2-7b-chat-int8 is capped at 50 requests per minute globally. A full list of current limits is maintained in the official documentation.
Planned pricing model
Billing will not begin on day one of the beta, but Cloudflare has shared its intended pricing structure to help developers plan. Users will be able to choose between two execution tiers:
- Regular Twitch Neurons (RTN) — runs wherever capacity is available at $0.01 per 1,000 neurons
- Fast Twitch Neurons (FTN) — runs at the nearest user location at $0.125 per 1,000 neurons
A neuron is Cloudflare’s unit for measuring AI output, and it scales to zero: if you have no usage, you pay nothing. For reference, 1,000 neurons roughly equates to 130 LLM responses, 830 image classifications, or 1,250 embeddings. The goal is to let developers pay strictly for what they use, choosing the tier that best balances cost against latency.
What’s on the roadmap
Cloudflare is positioning this beta as an early step and is soliciting feedback to shape future work. Several developments are already planned.
Expanded model catalog
The beta launches with a working set of models, but more will be added based on developer requests. Cloudflare is also partnering with Hugging Face, and a subset of the Hugging Face catalog will soon be directly accessible from Workers AI. Developers can submit model requests through the Cloudflare Developers Discord.
Analytics and observability
The current focus has been on making it easy to run models in a few lines of code. The next phase adds analytics and observability tools, giving developers per-model visibility into usage, performance, and spend, with the option to integrate into existing log pipelines for deeper exploration.
GPU rollout across the network
Cloudflare’s stated target is to be the best place for inference on “Region: Earth,” and the company is adding GPUs to its data centers as quickly as possible. The plan is to have GPUs in 100 data centers by the end of the current year, with near-comprehensive coverage by the end of 2024.
Developers can get started via the Workers AI documentation, and questions or project shares are welcome in the Developer Discord.
