
It's Christmas time and you've been wracking your brain trying to find the perfect gift that will teach your loved ones about the cloud in a simple and entertaining way. What to do? Fortunately for you, Santa has a new elf this year—Forrest Brazeal—who is part AWS Serverless Hero, part skilled cartoonist, and part cloud guru. Yes, it's a cartoon book about the cloud! The Read Aloud Cloud No, I did

At Netflix, our current data warehouse contains hundreds of Petabytes of data stored in AWS S3, and each day we ingest and create additional Petabytes. At this scale, we can gain a significant amount…

Hey, it's HighScalability time once again! NASA Do you like this sort of Stuff? Without your support on Patreon this Stuff won't happen. Know someone who could benefit from becoming one with the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 212 mostly 5 star reviews. Here's a load-balanced and fault-tolerant review: Number Stuff: * 1 billion: TikTok users in half the

Last year we deployed a CSAM image scanning tool. This is so cool! Image processing is always hard, and deploying a real image identification system at a Cloudflare scale is no small achievement! But we hit a problem - the matching algorithm was too slow for our needs.

Learn more about how we are bringing encapsulation to our views as we scale to over 4,500 templates in our Ruby on Rails monolith.

Who's Hiring? * Looking to rapidly hire Top Software Developers? Get Started with Toptal. Toptal will match you with top-quality, pre-screened freelance software developers that meet your project requirements. All in under 48 hours. Get started right away with a no risk trial. * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break

In this post, we’ll outline the approach we took to reliably scale our data platform in preparation for Black Friday and Cyber Monday.

Our aspiration to augment and improve every aspect of business and life with data, demands a paradigm shift in how we manage data at scale. While the technology advances of the past decade have addressed the scale of volume of data and data processing compute, they have failed to address scale in other dimensions: changes in the data landscape, proliferation of sources of data, diversity of data u

We cover our approaches to capacity planning, and how we rolled it out across the org and to dozens of teams. We’ll also share how we validated our capacity plans with scalability tests to make sure they work.

Presenting how Shopify got from no Sorbet in our monolith to almost full coverage in the span of a few months. I’ll explain the challenges we faced, the tools we built to solve them.

Hey, no power outages this week, so it's finally HighScalability time! Stunning: Tycho Crater Region with Colours by Alain Paillou Do you like this sort of Stuff? Without your support on Patreon this Stuff won't happen. Know someone who could benefit from becoming one with the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 189 mostly 5 star reviews. Here's a 100% lact

What do you do when you find a snake in your datacenter? You might say this. (NSFW) Fault tolerance is the property that enables a system to continue operating properly in the event of the failure of some of its components. You might think Facebook solved all of its fault tolerance problems long ago, but when a serpent enters the Edenic datacenter realm, even Facebook must consult the Tree of Know

It's not all that different really, especially that part where you can lose all your bitcoin. Here's an excerpt from East of Eden by John Steinbeck: Say, Carlton, how do you go about telegraphing money?” “Well, you bring me a hundred and two dollars and sixty cents and I send a wire telling the Valdosta operator to pay Adam one hundred dollars. You owe me sixty cents too.” “I’ll pay—say, how do I

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

Hey, it's HighScalability time! I can't wait for the duel. Just don't shoot into the air. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. Know someone who could benefit from becoming one with the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 167 mostly 5 star reviews. Here's a 100% lectin-free review:

Like many of those in tech, we invest heavily in our cloud and data infrastructure. While seemingly routine, the ability to manage and scale our infrastructure to support our 299+ million listeners worldwide, 24/7, without missing a beat (or syllable) is crucial for the business and our brand.

Stripe’s approach to creating a 1:40 million-scale, interactive 3D model of the earth.

Are you leading teams in a remote world? We are wrapping up our remote working series with some final tips on leading remote and distributed teams. We hope our teams have helped while we collectively work remotely from around the world.

Hey, it's HighScalability time! Serverless is really complex. Or is it? @paulbiggar sparked a thoughtful Twitter thread. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. Usually I pitch my book Explain the Cloud Like I'm 10 here, but if you're an author with books on Amazon I'd like to ask you to give Best Sellers Rank a try. I started it to send dat

Shopify recently launched the ability to add videos and 3d models to products as a native feature within the Admin section. Learn how we used pipelines to manage the long-term maintenance and understandability of the code.

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

Hey, it's HighScalability time! Line noise? Perl? Still uncertain? It's how you program a quantum computer. Silq. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world. Know someone who could benefit from understanding the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 ju

Now you can create custom workflow templates to promote best practices and consistency across your organization.

This is a guest post by Eunice Do, Data Engineer at TripleLift, a technology company leading the next generation of programmatic advertising. What is the name of your system and where can we find out more about it? The system is the data pipeline at TripleLift. TripleLift is an adtech company, and like most companies in this industry, we deal with high volumes of data on a daily basis. Over the la

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

Developers write tests to ensure correctness and allow future changes to be made safely. However, as the number of features grows, so does the number of tests. Tests are a double-edged sword. On one hand, well-written ones catch bugs and maintain a program’s stability, but as the code base grows, a high number of tests impedes scalability because they take a long time to run and increase the likel

Great logo design can do incredible things for the company behind it. Set the tone for the brand. Improve memorability. Help users emotionally connect to the brand. Create consistency between all marketing channels. Set it apart from the competition. But as a web designer, do you know how to tackle the lofty request of logo design when a client brings it to you?

Hey, it's HighScalability time! LOL. Who knew a birthday service could lead to an existential crisis? Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world. Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 111 mostly 5

Zoom scaled from 20 million to 300 million users virtually over night. What's incredible is from the outside they've shown little in the way of apparent growing pains, though on the inside it's a good bet a lot of craziness is going on. Sure, Zoom has made some design decisions that made sense as a small spunky startup that don't make a lot of sense as a defacto standard, but that's to be expected

This is the second part of my interview with Alex DeBrie on his new insta-classic: The DynamoDB Book. To read the first part of the interview please mosey on over to The DynamoDB Book: An Interview With Alex DeBrie On His New Book. Go ahead. Take your time. It's worth it. Quick Fire Round. What's your quick reaction to: The cost of storing data in S3 versus storing data in DynamoDB? Use S3 for lar

GitHub international leaders share their insights on how to work with globally distributed teams.

By: Jeet Mehta and Kathy Ge With over 1M business owners now on Shopify, there are billions of products being created and sold across the platform. Just like those business owners, the products that they sell are extremely diverse! Even when selling similar products, they tend to describe products very differently. One may describe their sock product as a “woolen long sock,” whereas another may ha

You know nothing about DynamoDB. At least that’s what I realized the first time I heard Rick Houlihan give his now infamous talk at AWS re:Invent 2018 on Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB. In that talk Rick revealed for the first time the inner arcana of single-table design. Minds were blown. Weaknesses were revealed. Futures were changed. As a mere novice in the way

There may be an undiscovered tribe deep in some jungle somewhere that hasn’t made up their mind on microservices, but I doubt it. People love microservices or love to hate microservices. There’s not much in between. So it means something when even a team at a company like Uber announces a change away from microservices to something else. What? Macroservices. But we’ll get to that. Think what you w

Firefox 72 was first out of the gate with "independent transforms." That is, instead of having to combine transforms together, like:

Hey, it's HighScalability time! Awesome explanation of how to build a PID controller to fly a rocket! (BPS.space via Orbital Index) Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world. Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon

In previous articles, we explained what consistency is, the difference between "strong" and "eventual" consistency, and why this distinction is more important

Author’s note: This is a copy of the white paper of the Working Group, a group of four pseudonymous professionals in Tokyo. I, Patrick McKenzie, was the primary author. It was distributed quietly during the week of March 25th, 2020. I have written an essay describing how this document came to be and demonstrating its provenance .

With easier organization and increased scalability, repository admins can more efficiently control access management to get users the permissions they need.

Hey, it's HighScalability time! The top 10,000 most spoken words in English represented by a point in hundreds of dimensions where the distance and direction between points encodes the relationship between words. (roadmaps) Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world. Know someone who w

In the previous article, we explained what strong (vs. eventual) consistency is. This article is the second part of a series where we explain how a lack of

Technical blog post on Dropbox's get_thumbnail API and when to use it. Includes analysis on file size and a sample script to follow along.

Years ago, we called Dropbox a “Magic Pocket” because it was designed to keep all your files in one convenient place. Dropbox has evolved from that simple beginning to become one of the most powerful and ubiquitous collaboration platforms in the world. And when our scale required building our own dedicated storage infrastructure, we named the project “Magic Pocket.” Two and a half years later, we’

More than ever, new products aim to make an impact on a global scale, and user experience is rapidly becoming the determining factor for whether they are successful or not.

I am delighted to share that I have joined Cloudflare as its first Chief Information Officer to help scale the company in this new phase of its business.

Wake up! It's HighScalability time: The Smithsonian has million of pieces of delicious open access content. I ate up this 3D representation of the 1903 Wright Flyer. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. That's how important you are to the fate of the world. Need to understand the cloud? Know someone who does? I wrote Explain the Cloud Lik

I was emailing a service about their documentation and while their doc was good, about one particularly tricky concept they told me that once you use it for a while, that’s when you’ll understand it. In other words: you’ll only understand it after you understand it. I didn’t like that response. I want documentation that takes me from an unproductive newbie to a somewhat functioning journeyperson.

Wake up! It's HighScalability time: Visualize the huge scale of Deep Time by identifying key reference points along the way. Do you like this sort of Stuff? Without your support on Patreon Stuff won't happen. I also wrote Explain the Cloud Like I'm 10 for everyone needing to understand the cloud (who doesn't?). On Amazon it has 93 mostly 5 star reviews (152 on Goodreads). Please be a real cloud he

How do we know if we are truly impactful as a design team? Are we seen as a vehicle to deliver a solution that moves a needle? The business value of design has been proven at scale by the McKinsey Design Index. and the study shows the best design performers increased their revenues and shareholder returns at nearly twice the rate of their industry counterparts. It’s time to empower our design team

I'm a fan of the componentization of the web. I think it's a very nice way to build a website at just about any scale (except, perhaps, the absolute most

Wake up! It's HighScalability time: Forrest Brazeal warns you not to spend your limited innovation credits building bespoke, complex systems that provide no direct value to your business, instead position yourself at the top of what heI calls the Wisdom/Cleverness Curve Do you like this sort of Stuff? Your support on Patreon is appreciated more than you can know. I also wrote Explain the Cloud Lik

No frills, or flashing neon frills with sprinklers attached? ‘Brutalist’ websites have flourished in recent years, but their guiding philosophy remains unclear. Brutalist web design has grown so quickly that there does not seem to be a clear consensus on what the style actually is. To some it means practicality, to others audacity. Love it or hate it, brutalist architecture celebrates rawness.






