Beyond deterministic automation

Traditional automation has a hard ceiling: it works beautifully as long as every input and edge case is anticipated in advance. The moment the data gets messy or the situation is unexpected, rigid scripts break down. Modern AI models change that equation by bringing judgement and reasoning into the execution loop, which is precisely what powers AI agents—systems that can decide which tools to use and how to sequence their work based on the context they encounter.

It helps to distinguish agents from the automation patterns that came before them:

  • Workflows: linear, deterministic execution paths that follow a fixed sequence.
  • Co-pilots: augmentative AI assistance that requires human intervention to complete tasks.
  • Agents: non-linear, non-deterministic systems that can adapt their approach from run to run.

Consider booking a vacation. A traditional workflow can take your dates, location, and budget, then call predefined APIs in order—but it stalls if a flight is sold out or a hotel is unavailable. A co-pilot can recommend itineraries and answer natural-language questions, but it can't execute the booking end to end. An agent, by contrast, combines AI's ability to make judgements with the capability to call tools. It can reassess when flights are unavailable, alter dates or locations, and continue working toward the goal on its own.

agents-sdk: a framework for production agents

With Cloudflare's new agents-sdk, you can add agent capabilities to an existing Workers project with a single command, or bootstrap a new project from the agents-starter template. Agents built with the SDK deploy directly to Cloudflare Workers.

The initial release is more than a thin wrapper around an existing library. Agents built with agents-sdk can communicate with clients in real time, persist state, execute scheduled long-running tasks, send emails, run asynchronous workflows, browse the web, query Postgres databases, call AI models, and support human-in-the-loop use cases—all out of the box.

For chat-based agents, the AIChatAgent class abstracts away much of the boilerplate. On the front end, a useAgent hook for React applications automatically establishes a bidirectional WebSocket, syncs client state, and lets you build agent-powered interfaces without a mountain of bespoke connection code.

The production story is a central part of the design. Agents run on top of Durable Objects, which function as stateful micro-servers that can scale to tens of millions of instances. This means an agent can run close to a user for low latency, close to the data it needs, or anywhere in between. The SDK also exposes:

  • State management APIs via this.setState plus a native sql API for querying data within each agent.
  • State synchronization between frontend applications and agent state.
  • Agent routing that enables agent-per-user or agent-per-workflow patterns—spawning millions of agents without manually provisioning CPU or scaling storage.

Planned additions include tighter email API integrations for human-in-the-loop flows, WebRTC hooks for voice and video interactivity, a built-in evaluation framework, and the option to self-host agents on your own infrastructure.

Controlled outputs: JSON mode in Workers AI

Agents depend on tool calling to translate natural-language requests into structured formats that APIs can process. For that to work reliably, text generation models must respond with valid JSON rather than free-form natural language. Workers AI now supports JSON mode, which lets applications request structured output from supported models.

JSON mode complies with OpenAI's response_format implementation. When a request includes a JSON schema definition, the model responds with an object validated against that schema. The feature is available on these models:

  • @cf/meta/llama-3.1-8b-instruct-fast
  • @cf/meta/llama-3.1-70b-instruct
  • @cf/meta/llama-3.3-70b-instruct-fp8-fast
  • @cf/deepseek-ai/deepseek-r1-distill-qwen-32b
  • @cf/meta/llama-3-8b-instruct
  • @cf/meta/llama-3.1-8b-instruct
  • @hf/nousresearch/hermes-2-pro-mistral-7b

JSON mode pairs naturally with function calling, and structured outputs work with traditional tool-calling patterns as well as through the Vercel AI SDK.

Context windows by token count

Workers AI is also changing how request sizes are limited. Previously, text generation requests were constrained by byte counts. The platform now measures by token counts, introducing explicit context windows and raising the limits for models in the catalog.

The context window is the total of input, reasoning, and completion or response tokens a model supports. Each model's documentation page now lists its context window limit so you can match model choice to your workload's requirements.

workers-ai-provider 0.1.1

The workers-ai-provider is Cloudflare's bridge between the popular AI SDK and Workers AI, letting you call Workers AI models with the same interface you'd use for any other LLM. Version 0.1.1 brings several improvements for agent builders:

  • Tool calling is now enabled for generateText.
  • Streaming works out of the box.
  • Usage statistics are enabled by default.
  • AI Gateway can be used even when streaming.

Agent workloads depend on fast model calls for routing, tool selection, and summarization, since those operations sit on the critical path of the user experience. Workers AI's globally distributed GPU fleet is well suited to the smaller, low-latency models that handle those tasks, and the updated provider makes it straightforward to use them with the development tools developers already know.

Cloudflare's bet on serverless fits the agent lifecycle

Cloudflare's platform philosophy, established with Workers in 2017, was built on the idea that applications should run as isolates on a global network—no regions, no concurrency management, no infrastructure scaling. That original bet on serverless computing turned out to align closely with the demands of AI agents, even if agents weren't the initial target. Many of the primitives Cloudflare shipped along the way, from Workers KV to Durable Objects, were designed for net-new use cases, but they map neatly onto the stop-and-go execution patterns agents exhibit.

Billing for CPU time, not wall-clock time

AI agents are fundamentally long-running tasks. They wait on slow reasoning models, poll external tools, and pause for human approvals. That creates a pattern of constant starting and stopping that is expensive if you pay for the entire duration of a process. Cloudflare Workers is built to scale down to zero and bills only for CPU time actually consumed, not wall-clock time.

The difference is often dramatic. A single LLM call might involve only 2–3 milliseconds of CPU activity while the wall-clock time stretches to 10 seconds. On Cloudflare's pricing model, that translates directly into savings for developers building agentic workflows.

Inference without idle GPUs

The same serverless logic applies to model inference. Foundation model providers offer straightforward APIs for calling hosted LLMs, but running open-source models, LoRAs, or self-trained models typically forces you onto traditional cloud providers that require pre-provisioning GPU capacity for peak traffic. That means paying for idle hardware the rest of the time.

Workers AI takes the opposite approach: you pay only when you call the inference API. There's no infrastructure to think about at all, which removes the capacity-planning burden from building agents that need on-demand model access.

Durable execution as an agent foundation

Agents often need to maintain state across requests and survive failures without losing context. Durable Objects and Workflows provide a programming model for guaranteed execution of asynchronous tasks that require persistence and reliability. They handle state maintenance across requests and automatic retries, which makes them well-suited for long-running LLM calls, human-in-the-loop approval gates, and interactions with flaky third-party APIs. That combination creates a resilient base for complex, multistep agent tasks that take significant time to complete.

Updated documentation and tooling

The Cloudflare agents documentation has been refreshed to cover the full range of what's discussed here, starting with the fundamentals of agents and moving into foundational examples of building with them.

The Workers prompt has also been updated with knowledge of the agents-sdk library. That means developers using Cursor, Windsurf, Zed, ChatGPT, or Claude can get assistance building AI agents and deploying them to Cloudflare directly from their existing tools.