Titus is Netflix's Kubernetes-based container platform, running millions of containers per week across tens of thousands of EC2 instances in multiple regions. By default many containers from many teams share each host, and that density is what makes the platform efficient. A small set of demanding workloads — including some of the most latency-critical real-time systems — cannot tolerate that sharing. Single-tenant Titus gives such a task its own VM while keeping the API, the deployment path, and the managed experience unchanged: the Compute/Platform team still owns and patches the host.

Where shared-kernel isolation stops working

Titus uses the kernel's own tools to separate tenants: cgroups for CPU, memory and disk; namespaces; seccomp and AppArmor; the Completely Fair Scheduler; the iocost controller; traffic shaping; plus per-container identity, IAM roles and security groups. These partition what the kernel can partition. They cannot partition what is shared in hardware — a cluster of cores shares a last-level cache and a memory path, so a neighbor pulling data from RAM evicts cache lines and slows a co-tenant even when neither exceeds a quota. There is no cgroup for memory bandwidth or cache residency.

A noisy neighbor and a co-located app each stay within their cgroup CPU and memory limits, but both draw on a shared last-level cache and memory bandwidth that lack cgroup fences. The noisy neighbor takes most of it, so the co-located app stalls and its tail latency rises even though no one exceeded a quota.

The platform has attacked this before. Predictive CPU isolation moved placement out of the kernel into a service that forecasts demand and re-pins containers to cores via combinatorial optimization solved every minute. Noisy Neighbor Detection with eBPF surfaces per-container run-queue latency in near real time. Mount Mayhem traced container-startup slowness to a global kernel lock amplified by modern CPU cache topology. Each raised the floor for everyone; none made co-tenancy invisible, because noisy neighbors are a family of problems, not one:

  • CPU performance indeterminacy. Co-located threads contend for the run queue even inside quota; large working sets evict last-level cache lines and saturate memory bandwidth, neither of which a CPU share can partition.
  • Network bandwidth and packets per second. Receive processing runs in softirq context, charging one container's packet work to a neighbor's CPU. A flood of small packets exhausts a core's softirq budget long before a bandwidth cap is reached.
  • Storage bandwidth and IOPS. A neighbor saturating a device queue serializes everyone's I/O, and cloud block storage imposes shared per-volume and per-instance ceilings.
  • Long tail. Page-cache reclaim, NUMA and cross-socket interconnect traffic, TLB-shootdown interrupts, PCIe and GPU contention, power and thermal throttling, and kernel global-lock contention each degrade neighbors differently.

Network contention has hurt most in practice. Per-container shaping through an HTB queueing discipline serializes traffic behind a global lock to account for each container's bytes and packets; a flood of small packets from one container burns kernel CPU on that lock and starves co-tenants of time to process their own traffic, producing TCP retransmits and tail-latency spikes on innocent services. The exposed workloads are stateful, high-throughput streaming jobs: real-time pipelines and stream processors. For these, interference is not a slowdown that averages out — a processor that falls behind may restore from a checkpoint and pause for many minutes, paging an on-call engineer. The same teams note, correctly, that they never saw this on a plain EC2 instance.

Options considered

VM isolation was the standard customers already expected. The question was how to offer it without losing what makes Titus useful. Four paths were weighed.

  1. Keep tuning shared-kernel isolation. It benefits every workload and needs no opt-in, and investment continues. Its limits are structural: each round narrows the gap without closing it, because cache, memory bandwidth and kernel shaping and scheduling locks are not fully containable by a container boundary. It is the right default and the wrong promise for a job that cannot tolerate a bad minute.
  2. microVMs (Firecracker, Kata Containers). Booting in about 150 milliseconds by emulating far less hardware, these were the most appealing alternative on paper. Three costs blocked broad adoption: hardware virtualization is available only on bare metal or through nested virtualization, the latter not ready on the instance types Netflix runs at the time, which would have confined the platform to a narrow slice of the fleet; there is no managed microVM service to build on, so the team would have to run a virtualization layer itself — hypervisor patching, live migration, device qualification; and microVMs add overhead on CPU- and IO-bound work and do not support GPUs.
  3. gVisor. A user-space kernel intercepting syscalls gives sandbox-grade isolation without a hypervisor, avoiding the bare-metal constraint. But syscall interception adds overhead on exactly the IO-heavy paths that need protection, and the open-source project lags its internal counterpart and is effectively maintained by a single vendor.
  4. Move the workload to EC2. Running containers directly on EC2 works, but the application team takes back instance selection, host patching, and fleet sizing and scaling. That is a regression for teams that came to Titus to avoid those things, and historically a one-way door.
Spectrum of isolation options from lighter and more shared to stronger isolation: shared-kernel containers, then gVisor, then microVMs (Firecracker or Kata), then a full VM as used by single-tenant Titus, which is highlighted. Each option is annotated with its launch latency, overhead, and the entity that operates it; single-tenant Titus is a dedicated EC2 instance with no additional virtualization overhead.

The threat model also matters. microVMs and gVisor target security isolation between mutually distrusting tenants. Among Netflix teams that is more than the situation warrants; what these workloads need is resource and performance isolation — a guarantee that a neighbor cannot steal their CPU, cache, memory bandwidth or network. A dedicated EC2 VM delivers that with hardware that needs no emulation and a kernel that needs no sandboxing, paying for the isolation needed rather than a stronger security boundary.

How the mode is built

A single-tenant task runs on its own EC2 instance, sized to fit, with nothing scheduled alongside — still a Titus container host owned and patched by the Compute/Platform team, reached through the same API and deployment path.

Side-by-side anatomy of a multi-tenant Titus, where multiple teams’ containers share one Linux kernel and the hardware beneath it and contend with one another, versus a single-tenant node where one task plus its platform sidecars owns the whole machine with no co-tenants.

Most of the work sits in the federation layer, which decides where a job runs before it is scheduled onto a node.

For an eligible job, three things happen.

  1. Federation decides eligibility and the envelope. Eligibility is governed by an allowlist the Compute/Platform team maintains; a team opts in by targeting a single-tenant workload group rather than flipping a flag in the public API. Federation tags the task as single-tenant and records the instance families it may run on, keeping performance consistent across the workload. Consolidating this in federation follows the platform's general routing principle: placement intelligence belongs in one place, above the individual cells, where it can account for capacity and availability across the region.
  2. Autoscaling provisions a dedicated node. For a pending single-tenant task, autoscaling launches, just in time, the smallest instance among the allowed families that satisfies the requested CPU, memory and GPU. Tightest fit keeps the unused remainder small.
  3. The scheduler enforces one task per node. Single-tenant pods carry pod anti-affinity against all other pods and a priority class exempting them from preemption. Nodes are tainted so unrelated workloads cannot land, with only single-tenant workloads tolerating the taint. Defragmentation and repacking are disabled for these nodes, so a task is never relocated to repack the fleet.

Two choices deserve attention. Allowlisting is deliberate: a free checkbox would be ticked by most teams, eroding the efficiency multi-tenancy provides, whereas central control directs dedicated isolation where it is needed and keeps the public API small while the mode matures. And jobs are never silently resized to fill their node. The job keeps what it requested; sizing close to a node shape is a recommendation. The workload stays portable and runs identically in either mode, so entering and leaving single tenancy is a configuration change, not a rewrite.

The economics: a few perpetrators, many victims

Noisy-neighbor pain is unevenly distributed. A small number of jobs flood the host with small packets, churn the page cache, or saturate memory bandwidth, and some of them are also among the most interference-sensitive — a job can be both source and victim. That shape makes single tenancy a precise lever: moving the handful of perpetrators and the most sensitive victims onto dedicated nodes protects their former neighbors and removes the need to spread sensitive workloads with anti-affinity placement. Hosts left behind run quieter and pack more densely, since they no longer need to stay partly empty to honor spread rules.

Before-and-after view of the Titus fleet. Before: a few red perpetrators are scattered across shared hosts, the amber workloads beside them are disrupted, and anti-affinity spreading leaves dashed stranded capacity. After: each perpetrator runs on its own dedicated single-tenant node, the remaining shared hosts are fully loaded with healthy workloads, and one host’s worth of capacity is reclaimed.

Each perpetrator gets a right-sized node where it can be as noisy as it likes; the remaining hosts fill with undisturbed workloads; stranded capacity comes back.

Provisioning many small nodes

Trading fewer large shared instances for more small dedicated ones shifts load across the stack.

The first consequence is control-plane load. More nodes mean more for the Titus control plane to track and more VM launches against the EC2 control plane — the latter mattering most during a regional failover, when scale-up is fastest and a throttle is least affordable. Titus is already federated: each region is split into independent cells that bound any single control plane's size and allow capacity to be added by adding cells. Workloads migrate in stages, and the launch-rate path is exercised under controlled load on a schedule rather than tested during an incident.

The second is capacity management, where the design changed most. With one task per node, the fleet spans many instance shapes whose demand shifts continuously. Pre-staging static pools of every shape would be slow to manage and wasteful, so single-tenant nodes are provisioned on demand:

  • Just-in-time, right-sized provisioning. Jobs are configured with a list of supported instance families based on performance requirements; at launch they map to the best-fit node size by comparing requested resources against those families, and the autoscaler brings up nodes of that type to meet pending demand. Anti-affinity to all other pods makes the scheduler land one pod per autoscaled node. Newly vacated nodes may be reused by pending deploys before automatic teardown. Node pools are shared across single-tenant workloads and describe a catalog of available families and sizes.
  • Single-tenant node pools. These may run in separate clusters or be shared with multitenant clusters, are configured with a list of supported instance-type permutations, and are pinned to a Titus container host AMI — the host image and its configuration are declared once and applied to every node created.
  • Consolidation management. The multitenant default of periodically repacking and relocating workloads does not apply. Only nodes vacated by scale-in or termination are reclaimed, with termination in two stages so pending workloads can reuse a node before it is fully terminated.
  • Drift-based patching. A new container host AMI can be applied as a "mutable" deploy to existing nodes without affecting hosted containers, or as an "immutable" deploy that changes the target AMI for newly created nodes. Many upgrades therefore need no disruption, though the fleet may still be rolled with users when required.

Launch latency and the limits of warm pools

On-demand provisioning does not make a node appear instantly. Time to serving has two parts: standing the node up — booting a fresh EC2 instance, then pulling and extracting the container image, which dominates host setup for most jobs — and application startup such as JVM class loading and JIT compilation or a cache fill. A fresh instance takes tens of seconds before the image pull even begins.

Multitenant Titus hides most of this: a new task usually lands on a running host with image layers cached, so start takes seconds rather than minutes, with the host boot amortized across every task that has run there. Single tenancy gives that up — every task waits for its own machine to boot and pull its image.

Warm pools are the usual remedy, and they work well for the multitenant fleet precisely because it is nearly homogeneous: a handful of large shapes covers almost everything, so a little warm capacity yields a high hit rate at a cost spread across many future tasks. Single tenancy inverts those economics.

With every task on a right-sized node, the fleet spans many families and sizes, and a warm instance helps only a pending task whose shape matches: a warm r7i.2xlarge does nothing for a task needing an m7a.8xlarge. A good hit rate would require idle capacity in every family-and-size bucket, and idle capacity in a bucket with no task is pure cost. Tight best-fit sizing, the very property that makes single tenancy efficient at steady state, works against warm-pool reuse. Pressure also peaks at the worst moment: during a regional failover or live event, many tasks need fresh nodes at once, a thin spread-out warm pool drains immediately, and the rest pay full cold-boot cost.

Warm capacity is therefore a small, targeted lever, not a default. A little is pre-warmed for the most latency-sensitive single-tenant starts, and the shape menu is narrowed to a curated set of sizes so warm capacity concentrates into fewer buckets, trading some best-fit efficiency for a better hit rate. Everything else provisions just in time and accepts the boot.

The more durable wins cut launch cost itself, and they help single- and multi-tenant nodes alike:

  • Faster image distribution. Since the pull dominates host setup, newer OCI image formats and lazy-pulling snapshotters such as stargz and SOCI — which fetch only the bytes read at startup — reduce the largest part of the gap.
  • Checkpoint and restore. Snapshotting a warmed-up application with a tool like CRIU and restoring it, rather than cold-starting, removes most of the second half of launch latency.
  • Instance type selection. Multitenant configurations have historically used metal instances for slightly better performance at the cost of slower startup; because node startup latency matters more under single tenancy, the platform has moved away from them.
  • Scheduling optimization. Stress testing of fleet scaling exposed a bottleneck in Kubernetes preemption logic: nodes scaled up quickly sat idle awaiting scheduling. Since single-tenant pools run no preemptible workloads, disabling preemption greatly improved scheduling under strain.

The tradeoff remains: a single-tenant node is slower to stand up than a slot on a warm shared host, and warm pools cost more per node here because they cannot be as homogeneous. For the long-running services targeted by the mode, paying once at launch for a quiet life is a trade those teams accept.

Measuring isolation

Isolation claims are easy to assert, so the platform measures them with a repeatable benchmark and a single score. The method is differential: a reference container, a healthy representative application, records throughput and tail latency while running alone, then runs again beside a havoc container built to be a bad neighbor — a UDP flood through XDP stressing the network-shaping path, pointer-chasing and STREAM to pollute the last-level cache and saturate memory bandwidth, fsync storms against disk, stress-ng abusing the scheduler, and GPU-burn monopolizing an accelerator. A run is trusted only once the host reaches steady state, confirmed through the kernel's Pressure Stall Information: cpu_some or mem_some must show tasks actually stalling on a contended resource, not merely that it is busy. Scenarios run many times with fixed seeds, reported as a median with a bootstrapped confidence interval, because noisy-neighbor distributions are heavy-tailed and averages mask the spikes that matter.

From the reference numbers, throughput efficiency is the fraction of operations per second retained under attack, and latency efficiency is the ratio of undisturbed P99 to P99 under pressure. The composite is a weighted blend, with a latency-sensitive preset weighting tail latency more heavily for interactive and streaming-path services. A perfectly isolated system scores 100%, and a single hard failure invalidates the run regardless of the average.

cgroups isolation is imperfect, and applications may see worse disk or network performance from sharing volumes or interfaces with a determined neighbor; for distributed workloads this yields measurable inconsistency depending on replica placement. Single tenancy isolates workloads that cause problems for neighbors and/or are sensitive to them — either as a stopgap until a regression is fixed, or permanently for load patterns not worth supporting in a shared environment.

Fleet efficiency

The expectation is that dedicating nodes consumes slack and lowers utilization. In aggregate it can move the other way. To protect sensitive workloads before single tenancy, the platform used anti-affinity placement, scattering them across the fleet — and spreading fights bin-packing, since a host kept partly empty cannot be filled by a workload that would otherwise fit. Moving those workloads to dedicated nodes removes the need for spreading and the fragmentation that comes with it.

Disruption-sensitive services are another source of fragmentation. Typically stateful with expensive recovery, they can lead to nodes being held up by pods with strict disruption requirements, preventing defrag and recompaction during off-peak hours — symptoms mirroring those of anti-affinity rules. Depending on magnitude, moving such workloads to single-tenant pools can produce equivalent or even improved placement once all costs are counted, and it makes cost easier to quantify: waste becomes single-tenant node overhead rather than un-attributable fleet-wide fragmentation.

Practice decided the question. In theory multi-tenant bin-packing has the higher ceiling, but it is not reached: AWS offers a discrete set of instance sizes, so a large shared host rarely matches the exact sum of requests placed on it, and at large-instance granularity those bin-packing holes are sizable. Choosing the closest fit from many shapes for a single workload captured more efficiency in practice than the higher theoretical ceiling delivered.

Container-host maintenance

A full multitenant host is hard to drain: relocation means moving many tenants at once, several with disruption budgets limiting how fast they can move, turning a routine patch into a coordinated evacuation. A single-tenant host runs one task with consolidation and defragmentation off, so draining it relocates only that workload, which can be handled automatically when a new AMI ships. The remaining multitenant hosts become easier to maintain too, because the most disruption-sensitive workloads are no longer stranded on them.

A setting, not a migration

A team moves a workload onto a dedicated node and back with a labeling change, without rewriting anything and without leaving Titus. The isolation decision becomes reversible — the opposite of the former one-way door to EC2. A team can try single tenancy for a quarter, measure it, and step back.

The contract is unchanged: the Compute/Platform team selects the instance type, patches the host, and sizes and scales the fleet; the application team keeps the same API, delivery pipeline, and observability. Single tenancy adds a point on the isolation axis within the existing managed experience rather than being a separate product.

When not to use a dedicated node

Two costs should be weighed first. A single-tenant task gets the machine minus overhead: every Titus container host, dedicated or shared, runs platform sidecars, system services, the kubelet and the host environment, costing a roughly fixed slice of CPU and a few gigabytes of memory. On a shared host that slice is paid once and amortized across many tasks; on a dedicated node one task bears it. Because the cost is fixed rather than proportional, a small dedicated node can lose tens of percent of its resources to overhead while the largest shapes see it round to a couple of percent.

Second, because a task lands on the smallest instance that fits, there is usually a gap between what it requested and the next shape available, and with no neighbors that gap is pure waste. The two effects compound: fixed overhead plus best-fit remainder set a ceiling on usable capacity per dedicated node, lowest exactly where isolation is reached for casually, on small instances. Coordination with the service owner can reduce the sizing waste.

The arithmetic gives the guidance: single tenancy pays off for large, long-running, performance-sensitive, disruption-sensitive workloads sized close to a node shape and run as fewer, larger replicas. It is wrong for small, bursty or highly elastic workloads, where overhead fraction is high, the node sits mostly idle, and multitenant packing is cheaper and scales faster. That is also why the mode is allowlisted rather than self-serve.

Adoption so far

The first adopters are the workloads that motivated the work: real-time data infrastructure — managed Apache Flink stream processing, streaming data connectors, and telemetry and logging pipelines — which were both the most frequent sources of network contention and the most damaged by it, given costly recovery and tight disruption budgets. On dedicated nodes, the class of problem is removed rather than mitigated: teams report stable operation after intermittent, hard-to-attribute interference, and jobs are no longer relocated by fleet repacking, eliminating a second, self-inflicted disruption. Latency-sensitive serving paths and some GraphQL and edge services have followed for predictable tails over a few points of bin-packing efficiency.

Rollout is ongoing. The set of eligible workloads widens as knowledge grows, with the largest production jobs serving as the stress test for scaling assumptions, while control-plane and EC2 launch-rate metrics are watched as the count of small nodes rises. Periodic stress tests exercise scaling ahead of real workload, and autoscaling and scheduling logic continue to improve.

What's next

On single tenancy: smarter pre-warming of common shapes to cut launch latency, a broader allowlist as scaling data holds up, and extension of the model beyond long-running services as cost attribution and capacity stories mature. On the multitenant side, which stays the default for most workloads, the biggest target is the network — replacing HTB's global-lock shaping with an eBPF-based Expected Departure Time approach that rate-limits by timestamping packets rather than contending on a lock, paired with a lower-overhead container network device. Hardware cache and memory-bandwidth partitioning (Intel RDT, AMD PQoS), hardware GPU partitioning (NVIDIA MIG) and continued work on the cgroup v2 IO controller are under evaluation; the benchmark ranks these by return on effort and shows customers in numbers that the shared fleet is getting quieter.

Longer term, the Platform teams want a single Compute API that takes intent rather than mechanism — how much isolation a workload needs, how fast it must launch, how long it can run without disruption, and its business domain — and makes placement and runtime choices with full visibility across the fleet.