Home/Distributed
Topic

Distributed

248 articles on Distributed.

13,309 articles
Distributed — What it Took to Scale Spotify’s Marketing Efforts

What it Took to Scale Spotify’s Marketing Efforts

At Spotify, we’ve tried to uphold the tradition of running one major holiday marketing campaign at the end of each year. The campaigns typically run on various platforms such as the Spotify Homepage and the Premium page.

JLJames La Broy, Senior Engineer·July 1, 2022Distributed
Distributed — Building Background Effects for Clips

Building Background Effects for Clips

Last September, Slack released Clips, allowing users to capture video, audio, and screen recordings in messages to help distributed teams connect and share their work. We’ve continued iterating on Clips since its release, adding thumbnail selection, background blur, and most recently, background image replacement. This blog post provides a deep dive into our implementation of…

AXAlbert Xing·June 7, 2022Distributed
Distributed — Scaling Slack’s Mobile Codebases: Modernization

Scaling Slack’s Mobile Codebases: Modernization

In the first two posts about the Duplo initiative, we described why we decided to revamp our mobile codebases, the initial phase to clean up tech debt, and our efforts to modularize our iOS and Android codebases (post 1, post 2). In this final post, we will discuss the last theme of the Duplo initiative,…

TSTracy Stampfli·May 4, 2022Distributed
Distributed — Handling Flaky Tests at Scale: Auto Detection & Suppression

Handling Flaky Tests at Scale: Auto Detection & Suppression

At Slack, the goal of the Mobile Developer Experience Team (DevXp) is to empower developers to ship code with confidence while enjoying a pleasant and productive engineering experience. We use metrics and surveys to measure productivity and developer experience, such as developer sentiment, CI stability, time to merge (TTM), and test failure rate. The DevXp…

APArpita Patel·April 5, 2022Distributed
Distributed — Scaling Slack’s Mobile Codebases: Modularization

Scaling Slack’s Mobile Codebases: Modularization

In the first post about the Duplo initiative, we discussed the reasons for launching a project to revamp Slack’s mobile codebases, and what we accomplished in Duplo’s initial Stabilization phase. This post will explore modularization, and then there will be a third post to describe how we modernized our codebase and the overall results of…

TSTracy Stampfli·March 28, 2022Distributed
Distributed — Introducing the GHES repository cache

Introducing the GHES repository cache

If you’re a GHES customer with heavy read traffic on your monorepo, check out the repository cache, especially if you have CI workloads distributed around the world.

MCMatt CooperMatt Cooper·March 22, 2022Distributed
Distributed — Comparing Quantiles at Scale in Online A/B Testing

Comparing Quantiles at Scale in Online A/B Testing

TL;DR: Using the properties of the Poisson bootstrap algorithm and quantile estimators, we have been able to reduce the computational complexity of Poisson bootstrap difference-in-quantiles confidence intervals enough to unlock bootstrap inference for almost arbitrary large samples. At Spotify, we can now easily calculate bootstrap confidence intervals for difference-in-quantiles in A/B tests with

MSMårten Schultzberg and Sebastian Ankargren·March 1, 2022Distributed
Distributed — Building Self-driving Kafka clusters using open source components

Building Self-driving Kafka clusters using open source components

In this article, I will talk about how Slack uses Kafka, and how a small-but-mighty team built and operationalized a self-driving Kafka cluster over the last four years to run at scale. Kafka is used at Slack as a pub-sub system, playing an essential role in the all-important Job Queue, our asynchronous job execution framework…

SKSuman Karumuri·February 11, 2022Distributed
Distributed — Building Infrastructure Platforms

Building Infrastructure Platforms

Infrastructure Platform teams enable organisations to scale delivery by solving common product and non-functional requirements with resilient solutions. This allows other teams to focus on building their own things and releasing value for their users. But nobody said building scalable platforms was easy... In this article Poppy and Chris explore 7 key principles which can help you build the right

PRPoppy Rowse and Chris Shepherd·February 9, 2022Distributed
Distributed — Stuff The Internet Says On Scalability For January 28th, 2022 - High Scalability -

Stuff The Internet Says On Scalability For January 28th, 2022 - High Scalability -

Never fear, HighScalability is here! Think your software diagram is complex? This is a single cell modeled using X-ray, nuclear magnetic resonance (NMR), and cryo-electron microscopy datasets. Gael McGill My Stuff: * Love this Stuff? I need your support on Patreon to keep this stuff going. * Know anyone who needs to fix their cloud-obliviousness? My book teaches them all they need to know about th

HSHigh Scalability·January 28, 2022Distributed
Distributed — From Chaos To System In Design Teams — Smashing Magazine

From Chaos To System In Design Teams — Smashing Magazine

Joining a team where there are no established design practices may feel disheartening. Does this situation resonate with you? If yes, then read on — as this article is sharing the author’s advice based on his personal experience and it will show you a way (although not easy) of facing the problem in a manner that will bring other benefits, beside order and consistency.

JJaviercuello·January 21, 2022Distributed
Distributed — Stabilize, Modularize, Modernize: Scaling Slack’s Mobile Codebases

Stabilize, Modularize, Modernize: Scaling Slack’s Mobile Codebases

When do you need to overhaul a large code base to address tech debt? What is the best way to address widespread inconsistencies and outdated patterns? How can you make significant architectural improvements to a complex application while still continuing to ship features? These were questions we grappled with at the beginning of 2020, when…

TSTracy Stampfli·January 12, 2022Distributed
Distributed — Scaling the Practice of Architecture, Conversationally

Scaling the Practice of Architecture, Conversationally

Architecture need not be a monologue; delivered top-down from the minds and mouths of a centralised few. This article describes another way to do architecture; as a series of conversations, driven by a decentralised and empowering decision-making technique, and supported by four learning and alignment mechanisms: Decision Records, Advisory Forum, Team-sourced Principles, and a Technology Radar

AHAndrew Harmel-Law·December 15, 2021Distributed
Distributed — Architecture of Max reHIT Workout - High Scalability -

Architecture of Max reHIT Workout - High Scalability -

Today I have the pleasure of announcing my new app—Max reHIT Workout—on Product Hunt. Max reHIT Workout is an exercise app that guides you through interval workouts. I won’t pitch the app here. I'll just say I’m proud of how it turned out and if you want an optimal algorithm for exercising, you might like it. I know I haven’t been writing much lately. That's because there’s been very little evolut

HSHigh Scalability·November 30, 2021Distributed
Distributed — Stuff The Internet Says On Scalability For October 4th, 2021 - High Scalability -

Stuff The Internet Says On Scalability For October 4th, 2021 - High Scalability -

Hey, HighScalability is here again! The circulatory system of the internet. @tylermorganwall Love this Stuff? I need your support on Patreon to keep this stuff going. Sorry for the long gap in posting, but I’ve been building a new app. I’m looking for testers for my new iOS fitness app: Max reHIT Workout. It guides you through proven reduced-exertion high-intensity interval workouts. If that inter

HSHigh Scalability·October 4, 2021Distributed
Distributed — Real-Time Communications at Scale

Real-Time Communications at Scale

We’re making it easier to build and scale real-time communications applications around open technologies, starting with WebRTC Components.

SASilverlock, AchielSilverlock, Achiel·September 30, 2021Distributed
Distributed — AWS Lambdas: Easy, Easier, Easiest

AWS Lambdas: Easy, Easier, Easiest

I'd say cloud functions are one of the most transformative technologies in the last bunch of years. They are (usually) cheap, scale well, secure in their

CCChris Coyier·September 3, 2021Distributed
Distributed — Data Lineage at Slack

Data Lineage at Slack

Reinventing how the world does work inevitably creates a lot of data. Each year, Slack’s scale has increased and the volume of data ingested and stored has kept pace. To make it possible to understand relationships within our data, we’ve invested heavily in an automated data lineage framework. This facilitates producer/consumer coordination, improves risk mitigation,…

SBSamuel Bock·August 17, 2021Distributed
Distributed — Evolution of search engines architecture - Algolia New Search Architecture Part 1 - High Scalability -

Evolution of search engines architecture - Algolia New Search Architecture Part 1 - High Scalability -

What would a totally new search engine architecture look like? Who better than Julien Lemoine, Co-founder & CTO of Algolia, to describe what the future of search will look like. This is the first article in a series. Search engines, and more generally, information retrieval systems, play a central role in almost all of today’s technical stacks. Information retrieval started in the beginning of com

HSHigh Scalability·August 2, 2021Distributed
Distributed — Stuff The Internet Says On Scalability For June 25th, 2021 - High Scalability -

Stuff The Internet Says On Scalability For June 25th, 2021 - High Scalability -

Hey, it's HighScalability time! Only listen if you want a quantum earworm for the rest of the day. Not your style? This is completely different. No, it’s even more different than that. Today in things that nobody stopped me from doing: The AWS Elastic Load Balancer Yodel Rag. pic.twitter.com/ocyVLf8WlUMay 28, 2021 Love this Stuff? I need your support on Patreon to help keep this stuff going. Do em

HSHigh Scalability·June 25, 2021Distributed
Distributed — Stripe Sessions 2021

Stripe Sessions 2021

Maximize conversion, scale globally, and introduce new revenue streams with new products from Stripe.

WGWill GaybrickWill Gaybrick·June 17, 2021Distributed
Distributed — Linux Kernel vs. Memory Fragmentation (Part I) - High Scalability -

Linux Kernel vs. Memory Fragmentation (Part I) - High Scalability -

(External) memory fragmentation is a long-standing Linux kernel programming issue. As the system runs, it assigns various tasks to memory pages. Over time, memory gets fragmented, and eventually, a busy system that is up for a long time may have only a few contiguous physical pages. Because the Linux kernel supports virtual memory management, physical memory fragmentation is often not an issue. Wi

NXNan Xiang·June 8, 2021Distributed
Distributed — Stuff The Internet Says On Scalability For April 30th, 2021 - High Scalability -

Stuff The Internet Says On Scalability For April 30th, 2021 - High Scalability -

Hey, HighScalability is back! This channel is the perfect blend of programming, hardware, engineering, and crazy. After watching you’ll feel inadequate, but in an entertained sort of way. Love this Stuff? I need your support on Patreon to keep this stuff going. Do employees at your company need to know about the cloud? My book will teach them all they need to know. Explain the Cloud Like I'm 10. O

HSHigh Scalability·April 30, 2021Distributed
Distributed — Scaling monorepo maintenance

Scaling monorepo maintenance

At GitHub, we serve some of the largest Git repositories on the planet. We also serve some of the fastest-growing repositories. Each day, the largest repositories we host become even…

TBTaylor BlauTaylor Blau·April 29, 2021Distributed
Distributed — Sponsored Post: Failover Conf!, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte - High Scalability -

Sponsored Post: Failover Conf!, Kinsta, Bridgecrew, IP2Location, StackHawk, InterviewCamp.io, Educative, Stream, Fauna, Triplebyte - High Scalability -

Fun and Interesting Events * Since we’re all feeling the fatigue of staring at screens all year, we’re switching things up at this year’s Failover Conf! This one-day conference will be featuring LIVE fireside chats and keynotes where you’ll be able to get your questions answered in real time. You’ll also hear from a variety of industry experts in our two panel discussions. And when you need a brea

HSHigh Scalability·April 7, 2021Distributed
Distributed — Customization vs. Configuration in Evolving Design Systems

Customization vs. Configuration in Evolving Design Systems

When a design system first starts out, the promise of visual consistency glows bright — the ideal product would have only one set of buttons, a unified typography scale, and elements that look the same no matter which designer made the design or which developer programmed them to be real and deployed.

CECharlie Eshleman·April 1, 2021Distributed
Distributed — Stuff The Internet Says On Scalability For March 17th, 2021 - High Scalability -

Stuff The Internet Says On Scalability For March 17th, 2021 - High Scalability -

Hey, HighScalability is here again! Reverse engineering an ancient analog computer is a detective story worth reading. A Model of the Cosmos in the ancient Greek Antikythera Mechanism. Do you love this Stuff? Without your encouragement on Patreon this Stuff won't stuffin’ stuff. Know someone who needs to know the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 262 most

HSHigh Scalability·March 17, 2021Distributed
Distributed — Engineering dependability and fault tolerance in a distributed system - High Scalability -

Engineering dependability and fault tolerance in a distributed system - High Scalability -

This is a guest post by Paddy Byers, Co-founder and CTO at Ably, a realtime data delivery platform. You can view the original article on Ably's blog. Users need to know that they can depend on the service that is provided to them. In practice, because from time to time individual elements will inevitably fail, this means you have to be able to continue in spite of those failures. In this article,

HSHigh Scalability·February 19, 2021Distributed
Distributed — Sponsored Post: 3T, Bridgecrew, Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: 3T, Bridgecrew, Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * Looking to rapidly hire Top Software Developers? Get Started with Toptal. Toptal will match you with top-quality, pre-screened freelance software developers that meet your project requirements. All in under 48 hours. Get started right away with a no risk trial. * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break

HSHigh Scalability·January 25, 2021Distributed
Distributed — Sponsored Post: Bridgecrew, Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: Bridgecrew, Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * Looking to rapidly hire Top Software Developers? Get Started with Toptal. Toptal will match you with top-quality, pre-screened freelance software developers that meet your project requirements. All in under 48 hours. Get started right away with a no risk trial. * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break

HSHigh Scalability·January 5, 2021Distributed
Distributed — Using One Cron Parser Everywhere With Rust and Saffron

Using One Cron Parser Everywhere With Rust and Saffron

Cron parsers are everywhere, each supporting their own extensions on the cron string format, which made consistency between parts of Cron Triggers difficult to achieve. Today we’re showing how we easily solved this problem with Rust without needing to write the same code twice.

ALAaron LoydAaron Loyd·December 25, 2020Distributed