Thirteen Years In, Cloudflare Settles on a Definition

Cloudflare turned thirteen on September 27, 2023, and marked the occasion with its usual week of product announcements. But before getting to those, the company took the opportunity to address something it says has taken years to nail down: its own identity. Having resisted labels like "security company," "network," or even the dreaded "CDN," Cloudflare now describes itself as a connectivity cloud.

The framing is straightforward. "Connectivity" speaks to the mission of linking people, data, and applications across any infrastructure. "Cloud" conveys that the platform is managed, programmatic, and scalable, with baked-in security and intelligence. Cloudflare's positioning explicitly contrasts this with the major public cloud providers, which it characterizes as optimized for data retention and lock-in—calling them "captivity clouds" that hold customer workloads rather than enabling movement between services.

Cloudflare sees a particularly strong pull for this model among AI startups. The company claims it is the most commonly used cloud provider in the AI startup ecosystem, used to move training data to regions with available GPU capacity. In that context, the week's announcements focus heavily on inference—the process of running prompts through trained models to generate output like text or image recognition.

A Third Location for Inference

Inference has historically run in one of two places, each with tradeoffs. On a user's local device, inference is fast, keeps data private, and works offline, but devices face storage constraints, and adding high-end GPUs drives up cost and battery drain. On a centralized public cloud, models like OpenAI's GPT-4 can run at massive scale, but nearly all AI GPU capacity sits in data centers in the US, making every query a cross-continental round trip. That introduces latency, raises cost, and puts personal data at risk by moving it far from its origin.

Cloudflare is positioning its global network as a third, intermediary tier. The company is deploying modern GPU resources across its points of presence, with a goal of running models in more than 100 cities across 40+ countries by the end of 2023. By the end of 2024, it says inference-tuned GPUs should be deployed in nearly every city in its network, placing inference within milliseconds of devices worldwide.

The operational reality of that rollout is blunt: Cloudflare staff are physically transporting NVIDIA GPU cards to data centers around the world to expand the network's compute capabilities. The aim is to give developers a middle path—expansive resources that can sit geographically close to end users, run a wide range of open model architectures, and bill only for compute time spent on inference.

To illustrate the use case, Cloudflare cites an unreleased customer building consumer wearables. Those devices require on-device intelligence for navigation and safety but must also remain affordable and power-efficient. Using Cloudflare Workers AI to offload inference work lets the devices offload that compute burden while retaining privacy and battery life—an approach the company frames as offering "nearly boundless resources" while running near the user.

Workers AI: Serverless Inference Points

Centered in the announcement is Workers AI, which brings a serverless execution model to AI inference. It runs across Cloudflare's GPU-enabled infrastructure, letting developers access a catalog of open-source language and image models through Cloudflare's existing API endpoints and ecosystem. The service is built to fit into the developer workflow that the company has cultivated over the past several years.

Developers start by choosing from a selection of pre-supplied models or configuring a custom one with the model's identifier or the tag of their own uploaded build. There are two ways to run inference: the HTTP API or the Workers binding. The Python client offers a pull for interfacing from existing environments, and Cloudflare provides cURL for direct command-line access. An example of moving from image generation to similarity search with a models API is detailed in the documentation. The service runs inside Cloudflare's network, where the company says it can adapt to the demands of a given moment rather than maintaining cold server capacity.

The platform's default execution side is standard WebAssembly runtimes, and the AI gateway layer supplies features like caching and rate limiting. Cloudflare notes that Workers AI doesn't require DevOps teams to manage dedicated infrastructure; it can scale from zero to many concurrent users, and deployment is a matter of configuration rather than server provisioning. The company reports that inference requests are handled across its network with a focus on minimizing latency and avoiding the requirement to send every request either to a specific home region or to a separate, centralized cloud for processing.

Keeping GPU Allocation Tight

Even given the expansion of GPU resources into more cities, a cloud of nearly 300 locations still needs to meter scarce hardware properly. Cloudflare says its systems queue requests based on GPU availability, placing tens of thousands of workers onto a single GPU when possible. Part of the product design includes a model placement policy: a developer's custom models can run region-by-region while older or less popular models have their resources scaled back. GPU capacity is scheduled around network and user demand, keeping general operation within the Workers AI platform and this year's overall availability scope.

Cloudflare stresses that Workers AI is not just for hobbyist experiments, but is designed for production workloads—citing GPU ports deployed across its network so that requests stay closer to the user's geography. Control planes in addition track feature limits, keeping the platform aligned with publicly documented scale.

The company pairs its technical roadmap with a nod to its history of using birthday-week launches to push forward structural change. Past efforts include an IPv6 gateway, free and universal encryption, the Workers platform, and R2 object storage to move beyond the "captivity clouds." This year's emergence of AI-specific product pillars follows the same pattern, according to the company—providing "a dramatic impact on AI development" by making model inference easier to build upon.

As a self-referential demo, Cloudflare closes its letter with a poem about its identity—"connectivity cloud" versus "captivity cloud"—generated using Llama 2 on Workers AI in a nearby data center.

Prompt: “Write a poem that talks about the connectivity cloud and contrast it with the captivity cloud”