Why Spotify Standardized Its CDN Stack

Spotify's streaming service delivers millions of tracks and countless other assets to more than 230 million monthly active users. Years of growth left the company managing multiple disjointed CDN solutions for non-audio content. Audio streaming ran on a mature multi-CDN setup with Akamai and AWS, but images, client updates, and other assets were being served through widely varying paths, adding complexity to the architecture and creating inefficiency across R&D.

To address this, Spotify formed a dedicated squad focused on standardizing CDN usage around Fastly's edge cloud platform. The team's mandate included automated tooling, governance, and support across the company's diverse engineering groups.

The Fragmentation Problem

Spotify's engineering culture emphasizes squad autonomy. With over 2,000 R&D employees organized into squads that make independent technology choices, fragmentation is a natural risk. Content delivery had become one of the most painful areas.

Squads served some content directly from readable buckets like AWS S3 or Google Cloud Storage, simply making a bucket public and dropping the URI into code. Others repurposed services built for specific tasks like image resizing or watermarking for entirely different functions. Some teams inherited CDN endpoints without full knowledge of their configurations. Meanwhile, the critical audio path remained separate and well-optimized for low latency and high bandwidth.

This decentralization made consistent performance, security, and monitoring difficult to guarantee across all content types.

Building SquadCDN on Fastly

The new CDN squad set out to solve this with a simple workflow for getting a new CDN service running quickly on Fastly, which Spotify had already used for years in its audio delivery. The team's responsibility was to create a way for internal squads to get deployed while maintaining oversight.

They settled on a self-service tool called SquadCDN—an internal pre-deployment review system. Any Spotify squad can use it to submit a YAML string describing the domain, origin, bucket, and configuration flags. The tool generates a pull request that the CDN squad reviews and approves before deployment.

Automation with APIs and VCL

Alongside the review workflow, the squad wanted a low-maintenance system for themselves. They used Fastly's APIs for straightforward configuration tasks—creating services, setting hostnames, configuring origins and logging endpoints. For more sophisticated logic, they used Fastly's VCL (Varnish Configuration Language) to handle caching, edge application logic, error handling, redirects, token identification, and personalization based on location, language, or device type.

Convincing Squads to Migrate

The new system appealed easily to teams building brand-new services. Existing CDN users were more reluctant—migration seemed like extra work. The CDN squad promoted the benefits of faster performance, delivery metrics, and logging, plus 24/7 monitoring of the full request path all the way to origin.

The team also found that proactively reaching out to help was essential. They identified outlying CDN endpoints and offered to modify code themselves so that owning squads could migrate more easily. Reducing the effort required from other teams proved crucial to widespread adoption.

Operational Lessons from the Rollout

Several pitfalls and best practices emerged during the standardization effort:

  • Move critical services first. After migrating a few minor services, the team shifted priority to the most important public-facing content: audio, video, cover art, and artist images. This also surfaced problems in legacy VCL configurations and their own automated pipeline.
  • Protect secrets in logs. The squad relied on Fastly's Edge Dictionaries to store sensitive key values that only VCL can reference. Administrators viewing VCL see only variables, never private data like passwords.
  • Respect API call limits. Even generous limits can cause deployment failures when too many API calls happen at once. Careful call planning keeps deployments reliable.
  • Verify before deploying. The CDN squad gives squads a test file that exposes configuration or bucket permission issues early in the process.
  • Disallow writes to origin buckets. A CDN endpoint must not allow put requests that could overwrite files via bucket write credentials. None of Spotify's services have needed write access to the origin.
  • Always smoke test. After deployment, the squad performs end-to-end checks over the internet, curling for a known file. Custom VCL ensures the file bypasses cache so the test reaches origin. If issues appear, an automatic rollback to the previous version starts debugging immediately.
  • Enforce good practices deliberately. Squads must explicitly request uninterrupted HTTP delivery, giving the CDN team a chance to determine whether it's the right use case. Teams are also asked to tag services handling personal data for GDPR compliance, and to promote sensible caching and purging defaults.

Results After Standardization

The new configuration approach has been adopted across more than 60 squads, representing over 20 percent of the R&D organization. More than 80 services are now serving content through Fastly using the templated workflows and automated tooling.

The infrastructure win, though, was as much organizational as technical. Engineers across Spotify can now ship CDN-backed services without deep involvement in the details, while the CDN squad maintains visibility, governance, and a maintainable system.