When serverless meets AI’s execution model
Serverless computing was engineered for a different era of applications: quick, stateless web transactions that complete in milliseconds. Large language model (LLM) workflows don’t fit that mold. An interaction with an LLM is a sustained, multi-step sequence — a function starts, sends a request to an inference API, then waits for a response that can take seconds or minutes to return.
That waiting period exposes a fundamental inefficiency. The function stays alive and billable while doing no work. If it times out before the response arrives, a new instance must be spun up just to relay the result back to the client. Each subsequent turn in a conversation typically launches yet another function, even if earlier instances are still warm.
Four assumptions in traditional serverless design conflict with LLM usage:
- Transaction timing: Serverless targets millisecond responses; LLM inference takes seconds or more.
- Execution patterns: Functions are built for isolated requests, not sequential conversational flows.
- Resource lifecycle: The model assumes fast completion, not long-lived interactions.
- Scaling model: Horizontal scaling suits short bursts, not continuous AI processing.
Serverless still excels at its intended jobs. LLM workloads, however, represent a computational paradigm that calls for infrastructure designed around sustained execution and intermittent activity.
A compute model built for conversational AI
Fluid compute addresses this mismatch by reducing total compute consumption: it reuses existing capacity before spinning up new resources. Instead of instantiating a fresh function per request, it scales within a single instance first — a many-to-one approach that avoids cold starts and extra overhead. Compute is shared dynamically rather than duplicated unnecessarily.
The practical effect is that one instance can handle multiple AI inference requests concurrently. Conversation flows remain uninterrupted while developers avoid paying for idle function time. Every second of execution contributes to actual processing rather than waiting.
Fluid also places compute resources in regions close to the data they serve, which optimizes both performance and consistency for resource-intensive AI workloads. The model aims to give LLM applications both scalability and efficiency: responsive, secure apps with predictable costs.
Security architecture for AI traffic
Fluid compute pairs its efficiency measures with a layered security model intended for sensitive AI workloads.
- Edge inspection: Requests route to the nearest Vercel Point of Presence (PoP), where the Vercel Firewall inspects them before they reach the functions router. This Layer 7 protection blocks application-level threats, including DoS attacks, and filters suspicious traffic.
- Isolated instances: Fluid compute instances are never exposed directly to the internet. They connect to the Vercel Functions router through secure, persistent TCP tunnels, keeping all invocations and responses within a controlled channel.
- Availability safeguards: Fluid automatically fails over across availability zones within a region. Enterprise customers additionally get multi-region failover capabilities to maintain uptime during localized disruptions.
Default compute for new projects
Fluid compute is now the default for new projects on Vercel and can be enabled for existing deployments through function settings. For AI teams, the value proposition is straightforward: full resource utilization, reduced infrastructure waste, and the capacity to support real-time inference and background tasks without sacrificing security or responsiveness.
