159 articles from Spotify Engineering.
At the moment, I’m living with my wife and step daughter in London and working from home like most people. Which means I get up around 8am, drink lots of coffee (way too much!) and log onto my computer an hour later – ready to check my messages and start getting my brain into thinking mode.
In this post we’ll discuss how Spotify optimized and sped up elements from our largest Dataflow job, , for using a technique called Sort Merge Bucket (SMB) join. We’ll present the design and implementation of SMB and how we incorporated it into our data pipelines.
TL;DR We just open sourced — a tool for Apple’s developer software, Xcode, that lets you collect, display, and track the valuable metrics hiding inside your team’s Xcode build logs. Are your build times improving or regressing? Which version of Xcode is slowest? Which hardware setup is fastest? XCMetrics makes it easy to find out all this and more. Made for iOS engineers, by iOS engineers, the too
What if, for some users, the very best Spotify is a little less Spotify? Spotify Lite started as an experiment that had to be proven, both from a technical and a product-market fit perspective. In 2017, we found that a significant portion of registrations in some of our fastest-growing markets were happening on Android devices much older than what we were used to seeing in North American and Europ
So you’ve read of our two-part series about the new Experimentation Platform we’ve built at Spotify, and now know why we decided to invest in a new platform. In Part II, you’ll get a more detailed look at how we assigned users to experiments, how we analyze results and ensure test integrity.
TL;DR — our framework for building smarter data pipelines for audio and other media processing. Based on Python and Apache Beam, helps our teams process Spotify’s massive catalog of music and podcasts, faster and more efficiently. We think Klio’s — and its ability to let anyone leverage modern cloud infrastructure and tooling — has the potential to unlock new possibilities in media and ML research
At Spotify we try to be as scientific as possible about how we build our products. Teams generate hypotheses that we test by running experiments — normally in the form of an A/B test — to learn what works and what doesn’t. The learnings give us insights and fuel new product ideas.
If you hear faint whooping in the background of your playlists today, it’s just us celebrating a new milestone for Spotify’s open source efforts: has accepted , our open source developer portal, as an early stage project in the . It’s just the first step in a longer journey with the CNCF, but it’s an important one for Spotify as it underlines our renewed commitment to open source — and developers
In 2019, we began planning for Spotify’s second engineering conference set to take place in April 2020. It was expected to have about 1,500 attendees from a dozen different cities — seven separate tracks running for three days jam-packed with talks and, of course, social events, similar to the year before. The event was organized by 16 engineers from different parts of Spotify with added support f
As we prepare to launch , we remind ourselves of the challenges we took on and lessons learned to make this year’s experience even better for audio fans, like you, around the world.
Every second more than 30,000 people around the world press play on the same song on Spotify. Imagine if you could somehow see these musical connections as they were happening and connecting us with people we’ve never met, miles apart.
Like many of those in tech, we invest heavily in our cloud and data infrastructure. While seemingly routine, the ability to manage and scale our infrastructure to support our 299+ million listeners worldwide, 24/7, without missing a beat (or syllable) is crucial for the business and our brand.
“His spice-induced visions show him a myriad of possible futures where humanity has become extinct and only one where humanity survives. He names this future ‘The Golden Path’ and resolves to bring it to fruition.”
Across Spotify, our teams diligently strive to fulfill our mission to “unlock the potential of human creativity by giving millions of creative artists the opportunity to live off their work, and billions of fans the opportunity to enjoy and be inspired by it”. As product managers in the Platform Developer Experience (PDX) Tribe, part of Spotify’s Technology Infrastructure Group, we focus on unlock
TL;DR The pursuit wasn’t always easy, but “putting data first” has helped Spotify dramatically improve the infrastructure team’s decision-making process while improving company productivity. Raul Herbster describes the benefits, especially for mobile DevOps teams.
Usually when I’m commuting into the office, I have to get up at 7am and catch the subway. But these days, I can sleep a little later – I make myself a tea or coffee and check my emails and Slack messages at the kitchen table. Then I get to work and start to review code and pull requests that have come in overnight from developers in the US timezones.
In our experience, big technology upgrade efforts or migrations (as we fondly call them at Spotify) gear up with a lot of momentum in the beginning but tend to get stuck in the mud over time, eventually leading to a long tail of systems still left running on older versions. This is one of the ways fragmentation creeps into your tech infrastructure, like weeds sprouting in a garden — left unattende
Discover how we shortened the feedback loop of pre-merge verification checks for our iOS applications by up to 75%.
Usually, I live in Brooklyn and start the day with a 45-minute commute to the office. But at the moment, things are very different – my partner and I are holed up at our little place in the woods near New Paltz, so now I just start the day with some yoga before jumping online for my first meeting. When we bought this house last September, we had no idea what a refuge it would turn out to be. So I
Engineer Carl Engström explains how lifting the 10,000 limit on Liked Songs enabled his team to address a much deeper issue and improve the listening experience of end users, whatever their device or network.
is Spotify’s homegrown developer portal. We’ve been using it internally for years, but only recently released the open source version. You can read more about what Backstage does, and our plans for its future, in our previous post, “”
I usually get up way earlier than this, because it’s a 20-minute subway ride to the office and I have three children to drop off at school en route. But right now, we’re all staying home for the foreseeable future and that means I can take things a little slower, get up a little later and have an extra cup of coffee with my wife in the mornings. It’s a silver lining in this crazy situation.
The , which now appears at the top of Spotify Home, is an important step forward in how listeners experience their favorite music and podcasts. Engineering Manager, Damien Tardieu, tells the story behind the launch of this exciting new feature and why the methodology chosen to support its development was the foundation for success.
An Architecture Decision Record (ADR) is a document that captures a decision, including the context of how the decision was made and the consequences of adopting the decision. At Spotify, a handful of teams use ADRs to document their decisions. One of these teams, The Creator Team, focuses on providing tools for creators to express themselves on Spotify, as well as access data about their content.
Podcasts are exploding in popularity. Since 2015, we’ve added hundreds of thousands of shows, and users are listening more and more. With the additions of acquisitions including Gimlet and Parcast, we have a whole host of expertly created content, and with the addition of DIY podcasting platform Anchor, now everyone has access to tools to create their own podcast and publish it to Spotify, so the
I like to start the day slowly – I’m not a morning person, so my husband kindly walks the dogs while I enjoy the peace and quiet, and wake myself up with a matcha or kombucha. After that, I’m more ready to get going and start to think about the day ahead of me.
Mornings start with a wake-up call from my three young children. I get them fed, dressed and ready for school. But first, I need an espresso – that’s absolutely necessary if I’m going to get anything done! I don’t really go online until I’ve dropped off the kids around 8.15am, then I hop on the subway and head downtown, checking my email and Slack messages en route.
Yesterday, we released the open source version of , our homegrown developer portal. And we learned a thing or two via the feedback we received. So, I wanted to take this opportunity to further explain what we’re trying to do with Backstage — and more importantly, what we want to give to the greater engineering community beyond Spotify.
I get up, have breakfast and am ready to leave the house by 7.30 am. Every day, I cycle the 12km to and from the office—it’s mostly on safe, car-free bike tracks, and I love having the time to clear my head and think freely. It doesn’t even matter what the weather’s like—I’ll cycle, rain or shine.
This week, we launched our ! Third-party apps can now connect to Spotify and manage a user’s podcast library, search our podcast catalog, or fetch detailed information about podcast shows and episodes.
Spotify built its business on flawless content delivery. Our streaming platform serves up more than 50 million tracks (plus an array of images and other assets) to more than 230 million monthly active users around the world — making us one of the world’s leading streaming services. With content that feels instant and immersive, we help our customers have the best experience possible with their fav
The is one of Spotify’s largest marketing and social campaigns of the year. It enables our users to see a detailed breakdown of their listening habits over the past year. Since 2019 was the end of the decade, we wanted to do something special for our users. As one of the few streaming platforms that has existed since before 2010, Spotify had a unique opportunity to be able to provide users with a
Tune in to Hans Dockter, CEO & Founder of Gradle, as he explains the emerging practice of Developer Productivity Engineering and how data and acceleration technologies can speed up and improve essential software development processes.
Time to wake up, grab a shower and get my train. I live in Newton, Massachusetts, which means a half-hour ride into work. And I spend that time bouncing around between various streaming and podcast platforms on my phone—half to entertain myself, half as market research. It’s a great opportunity to put myself in the shoes of someone who doesn’t work at Spotify, listen with fresh ears, and figure ou
Hear from John Allspaw – former CTO of Etsy and one of the leading members of the DevOps movement – as he explains how Resilience Engineering techniques can help us understand incidents in online software worlds in a broader and deeper way.
I never set an alarm clock these days—my children wake me up sometime before 7am and we get them breakfast, help them get dressed and all get ready for the day. One of the coolest things about where we live in Stockholm is that everything is within walking distance—I just cross the street to drop our younger kid at preschool, go two blocks down to drop the older one at their school, and then walk
At Spotify, we believe strongly in data-informed decision making. Whether we’re considering a big shift in our product strategy or we’re making a relatively quick decision about which track to add to one of our editorially-programmed playlists, data provides a foundation for sound decision making. An insight is a conclusion drawn from data that can help influence decisions and drive change. To ena
This article is based on the keynote given by Tony Jebara at TensorFlow World in Santa Clara, California, October 2019. You can watch the presentation.
I wake up early enough to fit in a quick, 30-minute work out—I either go for a run, lift weights or walk my dog, who’s a Jindo mix rescue dog from Korea called Mari. Then, I make my way to work—sometimes stopping for a coffee en route, or sometimes grabbing breakfast when I get to the office.