When Your Edge Infrastructure Outgrows Your Core
Building DevCycle, our feature flagging platform, surfaced an unusual problem: the Cloudflare Workers serving our API requests scale so effortlessly that they expose weaknesses in the traditional AWS infrastructure behind them. Our Workers can absorb 10x traffic spikes without breaking a sweat, but the Kubernetes clusters and load balancers handling the corresponding event data cannot. This imbalance pushed us to rethink our architecture and move more of our core logic onto the Workers platform.
From Taplytics to DevCycle
For nearly a decade, Taplytics has provided no-code A/B testing and feature flagging to large consumer-facing companies. That experience included running over 140 billion requests through Cloudflare Workers. When we set out to build DevCycle—a feature management tool designed specifically for engineering teams—we knew Workers would be central to the new architecture. DevCycle is built using DevCycle itself, which let us practice continuous deployment while developing the product.
A Config-First Architecture
DevCycle's design centers on publishing and consuming JSON configuration files for each project environment. The publishing side runs on AWS services, while Cloudflare handles all high-scale consumption of those configs. This separation keeps our AWS footprint simple and low-scale while pushing latency-sensitive traffic to the edge.
We moved as much logic as possible into our server-side SDKs using local bucketing, and we share a WASM library between the SDKs and our Workers. That shared code reduces the maintenance burden across SDKs and ensures consistent behavior everywhere. The result is that most feature flag requests never need to travel to our origin infrastructure, which fundamentally changes our cost structure and lets us serve customers at any scale.
The Events Pipeline Bottleneck
Feature management platforms don't control when customers' applications generate traffic. Push notification-driven mobile apps, for example, can cause instantaneous 10x spikes in API load. Traditional auto-scaling and load balancers simply cannot react quickly enough to handle that pattern. The choices are to over-provision clusters for worst-case scenarios, risk rate-limiting legitimate requests, or redesign for the load.
Our Workers handle the feature flag requests without issue, but every flag request generates a corresponding events request sent to our AWS-based ingestion pipeline. That pipeline consists of an events API running in Kubernetes, which publishes to Kafka and eventually lands in Snowflake. The Workers scale instantly; the events containers and load balancer do not. The infrastructure that worked for years is now the bottleneck because Workers work too well in comparison to EC2 instances and EKS.
Our solution is a new events Worker that absorbs the instantaneous load and writes directly to S3 via Kinesis Data Firehose, which Snowflake already ingests. This removes the Kubernetes and Kafka components from the critical path and gives us a predictable cost structure regardless of traffic patterns. We're also watching Cloudflare Queues with R2 as a potential future path once a Snowflake connector exists.
Designing Without a Co-Located Database
Workers deliver fast responses, infinite scaling, and strong uptime only if you design for them properly. The key constraint: you cannot assume direct access to a centralized SQL or NoSQL database on every request. If your Worker reaches back to a database for each request, your latency is tied to geographic distance and database performance, your scale is limited by connection counts, and your uptime depends on that external service.
Instead, we rely on data provided in the API request itself and on cacheable data distributed across Cloudflare's network. Cloudflare offers several products to support this model, each with tradeoffs:
- KV: a global, low-latency key-value store. Data reads within a Worker are limited by a 60-second minimum TTL, so you must tolerate some staleness.
- Durable Objects: provide global uniqueness and transactional storage, which lets you keep user-level data close to the end user. The interface differs from what developers familiar with SQL or NoSQL systems might expect.
- R2: stores large amounts of unstructured data using familiar S3 APIs, and Cloudflare's cache can provide low-latency access from Workers.
- D1: a serverless SQLite database for SQL-based workloads.
Being an open platform, Workers can also reach any publicly available database. For DevCycle's EdgeDB product, we use Macrometa to give customers access to their user data from within our Workers.
Costs That Scale With Usage
Moving workloads to Workers gives us infrastructure spending that scales 1:1 with request volume. We no longer maintain idle EC2 capacity for spikes that may never come, and our costs map directly to customer usage. This contrasts sharply with SaaS pricing based on peak usage or other metrics disconnected from actual infrastructure spend.
For DevCycle, abandoning the legacy AWS architecture with its high fixed costs for databases and caching layers has made our infrastructure roughly 5x more cost-efficient to operate.
What's Next
We'll keep pushing our core logic closer to users, either on Cloudflare's network or inside our SDKs. As the Cloudflare developer platform matures, we expect to integrate R2, Queues, and Durable Objects more deeply into DevCycle's architecture. The direction is clear: serverless computing at the edge is not just a complement to traditional infrastructure—it is the replacement.
