Voice joins the Agents SDK without a second stack

Text-first interaction has shaped how most of us work with AI agents, but the moments where an agent is genuinely useful aren’t always in front of a keyboard. The Cloudflare team has addressed this with an experimental voice pipeline for the Agents SDK, released as @cloudflare/voice. It bolts voice onto the existing Agent architecture rather than forcing a migration to a separate framework.

A voice-enabled agent remains the same Durable Object, shares the same tools, and uses the same WebSocket connection model and SQLite conversation history. The @cloudflare/voice package ships with:

  • withVoice(Agent) for full conversational voice agents
  • withVoiceInput(Agent) for speech-to-text-only cases like dictation or voice search
  • useVoiceAgent and useVoiceInput React hooks
  • VoiceClient for non-React, framework-agnostic clients
  • Workers AI providers for out-of-the-box use without extra API keys, including Deepgram Flux and Nova 3 for continuous STT, and Deepgram Aura for TTS

Minimal wiring, streaming out of the box

The server-side pattern for a voice agent is intentionally lean. You attach a transcriber, plug in a TTS provider, and implement onTurn():

import { Agent, routeAgentRequest } from "agents";
import {
  withVoice,
  WorkersAIFluxSTT,
  WorkersAITTS,
  type VoiceTurnContext
} from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
  transcriber = new WorkersAIFluxSTT(this.env.AI);
  tts = new WorkersAITTS(this.env.AI);

  async onTurn(transcript: string, context: VoiceTurnContext) {
    return `You said: ${transcript}`;
  }
}

export default {
  async fetch(request: Request, env: Env) {
    return (
      (await routeAgentRequest(request, env)) ??
      new Response("Not found", { status: 404 })
    );
  }
} satisfies ExportedHandler<Env>;

Connecting from a React application is equally direct with the useVoiceAgent hook. If React isn’t your stack, VoiceClient from @cloudflare/voice/client works as a standalone.

BLOG-3198 2

The pipeline itself follows a compact seven-step loop that stays on one WebSocket connection:

  1. Audio transport: the browser sends 16 kHz mono PCM over the agent’s existing WebSocket.
  2. STT session: a continuous transcriber session starts when the call starts and lives for its duration.
  3. STT input: audio streams continuously into that session.
  4. Turn detection: the STT model itself decides when a user utterance is complete and emits a stable transcript.
  5. Application logic: the pipeline hands the transcript to your onTurn() method.
  6. TTS output: your response is synthesized and returned to the client. Streaming responses are sentence-chunked and sent as soon as each sentence is ready.
  7. Persistence: user and agent messages are stored in SQLite, surviving reconnects and redeployments.

Latency is a network path, not just a model problem

Most of the perceived lag in voice agents isn’t pure model inference time; it’s the overhead of moving audio and text between independent services. With the voice pipeline sitting on Cloudflare’s network and built-in providers pointing at Workers AI bindings, those hops get shorter. The pipeline also addresses time-to-first-audio by synthsizing and streaming each sentence of an onTurn() response as soon as it completes, so playback begins while the rest is still generating.

A fuller backend using an LLM demonstrates this well:

import { Agent, routeAgentRequest } from "agents";
import {
  withVoice,
  WorkersAIFluxSTT,
  WorkersAITTS,
  type VoiceTurnContext
} from "@cloudflare/voice";
import { streamText } from "ai";
import { createWorkersAI } from "workers-ai-provider";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
  transcriber = new WorkersAIFluxSTT(this.env.AI);
  tts = new WorkersAITTS(this.env.AI);

  async onTurn(transcript: string, context: VoiceTurnContext) {
    const ai = createWorkersAI({ binding: this.env.AI });

    const result = streamText({
      model: ai("@cf/cloudflare/gpt-oss-20b"),
      system: "You are a helpful voice assistant. Be concise.",
      messages: [
        ...context.messages.map((m) => ({
          role: m.role as "user" | "assistant",
          content: m.content
        })),
        { role: "user" as const, content: transcript }
      ],
      abortSignal: context.signal
    });

    return result.textStream;
  }
}

export default {
  async fetch(request: Request, env: Env) {
    return (
      (await routeAgentRequest(request, env)) ??
      new Response("Not found", { status: 404 })
    );
  }
} satisfies ExportedHandler<Env>;

Here, Context.messages exposes the SQLite-backed conversation history, and context.signal enables aborting the LLM call on user interruption.

One continuous state across input modes

A user can type, switch to voice, and drop back to text without ever leaving the same agent instance. The hooks blur the channel boundary: sendText() bypasses STT and delivers text directly to onTurn(). During a call, responses can be spoken and shown; outside one, they’re text-only. There’s no code-path split and no separate conversation store.

Standard agent features don’t disappear

Voice is an interface layer, not a walled-off service. Session-start greetings, scheduled spoken reminders, and LLM-exposed tools all remain available on the same agent.

import { Agent, type Connection } from "agents";
import { withVoice, WorkersAIFluxSTT, WorkersAITTS } from "@cloudflare/voice";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
  transcriber = new WorkersAIFluxSTT(this.env.AI);
  tts = new WorkersAITTS(this.env.AI);

  async onTurn(transcript: string) {
    return `You said: ${transcript}`;
  }

  async onCallStart(connection: Connection) {
    await this.speak(connection, "Hi! How can I help you today?");
  }
}

import { Agent } from "agents";
import {
  withVoice,
  WorkersAIFluxSTT,
  WorkersAITTS,
  type VoiceTurnContext
} from "@cloudflare/voice";
import { streamText, tool } from "ai";
import { createWorkersAI } from "workers-ai-provider";
import { z } from "zod";

const VoiceAgent = withVoice(Agent);

export class MyAgent extends VoiceAgent<Env> {
  transcriber = new WorkersAIFluxSTT(this.env.AI);
  tts = new WorkersAITTS(this.env.AI);

  async speakReminder(payload: { message: string }) {
    await this.speakAll(`Reminder: ${payload.message}`);
  }

  async onTurn(transcript: string, context: VoiceTurnContext) {
    const ai = createWorkersAI({ binding: this.env.AI });

    const result = streamText({
      model: ai("@cf/cloudflare/gpt-oss-20b"),
      messages: [
        ...context.messages.map((m) => ({
          role: m.role as "user" | "assistant",
          content: m.content
        })),
        { role: "user" as const, content: transcript }
      ],
      tools: {
        set_reminder: tool({
          description: "Set a spoken reminder after a delay",
          inputSchema: z.object({
            message: z.string(),
            delay_seconds: z.number()
          }),
          execute: async ({ message, delay_seconds }) => {
            await this.schedule(delay_seconds, "speakReminder", { message });
            return { confirmed: true };
          }
        })
      },
      abortSignal: context.signal
    });

    return result.textStream;
  }
}

You can also swap transcription models at runtime on a per-connection basis by overriding createTranscriber(). A common split would run Flux for conversational turn-taking and Nova 3 for accuracy-heavy dictation. Clients can pass the matching query parameters through the hook.

import { Agent, type Connection } from "agents";
import {
  withVoice,
  WorkersAIFluxSTT,
  WorkersAINova3STT,
  WorkersAITTS,
  type Transcriber
} from "@cloudflare/voice";

export class MyAgent extends VoiceAgent<Env> {
  tts = new WorkersAITTS(this.env.AI);

  createTranscriber(connection: Connection): Transcriber {
    const url = new URL(connection.url ?? "http://localhost");
    const model = url.searchParams.get("model");
    if (model === "nova-3") {
      return new WorkersAINova3STT(this.env.AI);
    }
    return new WorkersAIFluxSTT(this.env.AI);
  }
}

const voiceAgent = useVoiceAgent({
  agent: "my-voice-agent",
  query: { model: "nova-3" }
});

Hooks, telephony, and transport flexibility

To let developers intercept the flow between stages, the pipeline offers three hooks:

  • afterTranscribe(transcript, connection)
  • beforeSynthesize(text, connection)
  • afterSynthesize(audio, text, connection)

These slots are useful for content filtering, text normalization, language transforms, or custom logging.

Phone calls by Twilio adapter

The single WebSocket model isn’t the only path. A Twilio adapter connects traditional phone calls to the same agent:

import { TwilioAdapter } from "@cloudflare/voice-twilio";

export default {
  async fetch(request: Request, env: Env) {
    if (new URL(request.url).pathname === "/twilio") {
      return TwilioAdapter.handleRequest(request, env, "MyAgent");
    }

    return (
      (await routeAgentRequest(request, env)) ??
      new Response("Not found", { status: 404 })
    );
  }
};

That lets the one agent handle browser voice, typed text, and a dialed-in caller. One caveat for telephony use: the default Workers AI TTS brings MP3, but Twilio expects mulaw 8kHz audio. A TTS provider that outputs PCM or mulaw directly is the better choice there.

WebRTC with SFU support

For rough network conditions or multiparty scenarios, the voice package includes SFU utilities and accepts custom transports. The default line is WebSocket-native now, with adapters into Cloudflare’s global SFU infrastructure planned.

Small interfaces, interoperable ecosystem

The reason @cloudflare/voice can stay flexible is the compact shape of its provider contracts. A transcriber opens a continuous session and ingests audio frames; a TTS provider takes text and returns audio and, ideally, streams it back.

interface Transcriber {
  createSession(options?: TranscriberSessionOptions): TranscriberSession;
}

interface TranscriberSession {
  feed(chunk: ArrayBuffer): void;
  close(): void;
}

interface TTSProvider {
  synthesize(text: string, signal?: AbortSignal): Promise<ArrayBuffer | null>;
}

Built-in Workers AI providers keep the default path frictionless:

  • WorkersAIFluxSTT for conversational streaming STT
  • WorkersAINova3STT for dictation streaming STT
  • WorkersAITTS for text-to-speech

Because the interfaces are small, third-party services can implement them without touching SDK internals. Providers capable of accepting streaming audio and detecting utterance boundaries can satisfy the transcriber interface; a TTS provider that streams output is an unqualified fit. Cloudflare is courting STT players like AssemblyAI, Rev.ai, and Speechmatics; TTS names like PlayHT, LMNT, Cartesia, Coqui, Amazon Polly, and Google Cloud TTS; telephony connectors for Vonage, Telnyx, and Bandwidth; and bridges to WebRTC data channels and other transports. Collaborative work is also on the table, including cross-vendor latentency benchmarking, non-English language support, and accessibility work around multimodal interfaces.

Experimental now, open from day one

The package is available today as experimental. You can add @cloudflare/voice, assign a transcriber and TTS provider to your agent, deploy, and start talking to it. API details are covered in the reference docs, and ideas or issues can go to github.com/cloudflare/agents. The goal is structural, not cosmetic: voice should not demand a dedicated stack, and agents should be durable and unified no matter how the user speaks to them.

BLOG-3198 3