Home/Distributed
Topic

Distributed

248 articles on Distributed.

13,309 articles
Distributed — The Read Aloud Cloud: An Interview With Forrest Brazeal On His New Book - High Scalability -

The Read Aloud Cloud: An Interview With Forrest Brazeal On His New Book - High Scalability -

It's Christmas time and you've been wracking your brain trying to find the perfect gift that will teach your loved ones about the cloud in a simple and entertaining way. What to do? Fortunately for you, Santa has a new elf this year—Forrest Brazeal—who is part AWS Serverless Hero, part skilled cartoonist, and part cloud guru. Yes, it's a cartoon book about the cloud! The Read Aloud Cloud No, I did

HSHigh Scalability·December 23, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For December 19th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For December 19th, 2020 - High Scalability -

Hey, it's HighScalability time once again! NASA Do you like this sort of Stuff? Without your support on Patreon this Stuff won't happen. Know someone who could benefit from becoming one with the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 212 mostly 5 star reviews. Here's a load-balanced and fault-tolerant review: Number Stuff: * 1 billion: TikTok users in half the

HSHigh Scalability·December 19, 2020Distributed
Distributed — Computing Euclidean distance on 144 dimensions

Computing Euclidean distance on 144 dimensions

Last year we deployed a CSAM image scanning tool. This is so cool! Image processing is always hard, and deploying a real image identification system at a Cloudflare scale is no small achievement! But we hit a problem - the matching algorithm was too slow for our needs.

MMMarek MajkowskiMarek Majkowski·December 18, 2020Distributed
Distributed — Sponsored Post: Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: Toptal, IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * Looking to rapidly hire Top Software Developers? Get Started with Toptal. Toptal will match you with top-quality, pre-screened freelance software developers that meet your project requirements. All in under 48 hours. Get started right away with a no risk trial. * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break

HSHigh Scalability·December 8, 2020Distributed
Distributed — Data Mesh Principles and Logical Architecture

Data Mesh Principles and Logical Architecture

Our aspiration to augment and improve every aspect of business and life with data, demands a paradigm shift in how we manage data at scale. While the technology advances of the past decade have addressed the scale of volume of data and data processing compute, they have failed to address scale in other dimensions: changes in the data landscape, proliferation of sources of data, diversity of data u

ZDZhamak Dehghani·December 3, 2020Distributed
Distributed — Capacity Planning at Scale - Shopify

Capacity Planning at Scale - Shopify

We cover our approaches to capacity planning, and how we rolled it out across the org and to dozens of teams. We’ll also share how we validated our capacity plans with scalability tests to make sure they work.

SEShopify Engineering·December 3, 2020Distributed
Distributed — Adopting Sorbet at Scale - Shopify

Adopting Sorbet at Scale - Shopify

Presenting how Shopify got from no Sorbet in our monolith to almost full coverage in the span of a few months. I’ll explain the challenges we faced, the tools we built to solve them.

SEShopify Engineering·November 19, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For November 6th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For November 6th, 2020 - High Scalability -

Hey, no power outages this week, so it's finally HighScalability time! Stunning: Tycho Crater Region with Colours by Alain Paillou Do you like this sort of Stuff? Without your support on Patreon this Stuff won't happen. Know someone who could benefit from becoming one with the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 189 mostly 5 star reviews. Here's a 100% lact

HSHigh Scalability·November 6, 2020Distributed
Distributed — Snakes in a Facebook Datacenter - High Scalability -

Snakes in a Facebook Datacenter - High Scalability -

What do you do when you find a snake in your datacenter? You might say this. (NSFW) Fault tolerance is the property that enables a system to continue operating properly in the event of the failure of some of its components. You might think Facebook solved all of its fault tolerance problems long ago, but when a serpent enters the Edenic datacenter realm, even Facebook must consult the Tree of Know

HSHigh Scalability·September 22, 2020Distributed
Distributed — How did they send money in the olden days of telegraphs? - High Scalability -

How did they send money in the olden days of telegraphs? - High Scalability -

It's not all that different really, especially that part where you can lose all your bitcoin. Here's an excerpt from East of Eden by John Steinbeck: Say, Carlton, how do you go about telegraphing money?” “Well, you bring me a hundred and two dollars and sixty cents and I send a wire telling the Valdosta operator to pay Adam one hundred dollars. You owe me sixty cents too.” “I’ll pay—say, how do I

HSHigh Scalability·September 21, 2020Distributed
Distributed — Sponsored Post: IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: IP2Location, Ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

HSHigh Scalability·September 18, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For Sep 18th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For Sep 18th, 2020 - High Scalability -

Hey, it's HighScalability time! I can't wait for the duel. Just don't shoot into the air. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. Know someone who could benefit from becoming one with the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 167 mostly 5 star reviews. Here's a 100% lectin-free review:

HSHigh Scalability·September 18, 2020Distributed
Distributed — Managing Clouds from the Ground Up: Cost Engineering at Spotify

Managing Clouds from the Ground Up: Cost Engineering at Spotify

Like many of those in tech, we invest heavily in our cloud and data infrastructure. While seemingly routine, the ability to manage and scale our infrastructure to support our 299+ million listeners worldwide, 24/7, without missing a beat (or syllable) is crucial for the business and our brand.

JAJanisa Anandamohan·September 1, 2020Distributed
Distributed — Remote work: Sharing tips for leading in a remote world

Remote work: Sharing tips for leading in a remote world

Are you leading teams in a remote world? We are wrapping up our remote working series with some final tips on leading remote and distributed teams. We hope our teams have helped while we collectively work remotely from around the world.

LHLaura HeismanLaura Heisman·August 7, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For July 31st, 2020 - High Scalability -

Stuff The Internet Says On Scalability For July 31st, 2020 - High Scalability -

Hey, it's HighScalability time! Serverless is really complex. Or is it? @paulbiggar sparked a thoughtful Twitter thread. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. Usually I pitch my book Explain the Cloud Like I'm 10 here, but if you're an author with books on Amazon I'd like to ask you to give Best Sellers Rank a try. I started it to send dat

HSHigh Scalability·July 31, 2020Distributed
Distributed — Media at Scale: Callbacks vs pipelines - Shopify

Media at Scale: Callbacks vs pipelines - Shopify

Shopify recently launched the ability to add videos and 3d models to products as a native feature within the Admin section. Learn how we used pipelines to manage the long-term maintenance and understandability of the code.

SEShopify Engineering·July 9, 2020Distributed
Distributed — Sponsored Post: ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: ipdata, StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

HSHigh Scalability·July 8, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For June 26th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For June 26th, 2020 - High Scalability -

Hey, it's HighScalability time! Line noise? Perl? Still uncertain? It's how you program a quantum computer. Silq. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world. Know someone who could benefit from understanding the cloud? Of course you do. I wrote Explain the Cloud Like I'm 10 ju

HSHigh Scalability·June 26, 2020Distributed
Distributed — How TripleLift Built an Adtech Data Pipeline Processing Billions of Events Per Day - High Scalability -

How TripleLift Built an Adtech Data Pipeline Processing Billions of Events Per Day - High Scalability -

This is a guest post by Eunice Do, Data Engineer at TripleLift, a technology company leading the next generation of programmatic advertising. What is the name of your system and where can we find out more about it? The system is the data pipeline at TripleLift. TripleLift is an adtech company, and like most companies in this industry, we deal with high volumes of data on a daily basis. Over the la

HSHigh Scalability·June 15, 2020Distributed
Distributed — Sponsored Post: StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Sponsored Post: StackHawk, InterviewCamp.io, Educative, Triplebyte, Stream, Fauna - High Scalability -

Who's Hiring? * InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform. * Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n).

HSHigh Scalability·June 11, 2020Distributed
Distributed — Spark Joy by Running Fewer Tests - Shopify

Spark Joy by Running Fewer Tests - Shopify

Developers write tests to ensure correctness and allow future changes to be made safely. However, as the number of features grows, so does the number of tests. Tests are a double-edged sword. On one hand, well-written ones catch bugs and maintain a program’s stability, but as the code base grows, a high number of tests impedes scalability because they take a long time to run and increase the likel

SEShopify Engineering·June 11, 2020Distributed
Distributed — How To Design A Brand Logo (With Ease) — Smashing Magazine

How To Design A Brand Logo (With Ease) — Smashing Magazine

Great logo design can do incredible things for the company behind it. Set the tone for the brand. Improve memorability. Help users emotionally connect to the brand. Create consistency between all marketing channels. Set it apart from the competition. But as a web designer, do you know how to tackle the lofty request of logo design when a client brings it to you?

SSSuzanne ScaccaSuzanne Scacca·May 22, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For May 14th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For May 14th, 2020 - High Scalability -

Hey, it's HighScalability time! LOL. Who knew a birthday service could lead to an existential crisis? Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the smart and thoughtful world. Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 111 mostly 5

HSHigh Scalability·May 14, 2020Distributed
Distributed — A Short on How Zoom Works - High Scalability -

A Short on How Zoom Works - High Scalability -

Zoom scaled from 20 million to 300 million users virtually over night. What's incredible is from the outside they've shown little in the way of apparent growing pains, though on the inside it's a good bet a lot of craziness is going on. Sure, Zoom has made some design decisions that made sense as a small spunky startup that don't make a lot of sense as a defacto standard, but that's to be expected

HSHigh Scalability·May 14, 2020Distributed
Distributed — Part 2: The DynamoDB Book: An Interview With Alex DeBrie On His New Book - High Scalability -

Part 2: The DynamoDB Book: An Interview With Alex DeBrie On His New Book - High Scalability -

This is the second part of my interview with Alex DeBrie on his new insta-classic: The DynamoDB Book. To read the first part of the interview please mosey on over to The DynamoDB Book: An Interview With Alex DeBrie On His New Book. Go ahead. Take your time. It's worth it. Quick Fire Round. What's your quick reaction to: The cost of storing data in S3 versus storing data in DynamoDB? Use S3 for lar

HSHigh Scalability·May 4, 2020Distributed
Distributed — Categorizing Products at Scale - Shopify

Categorizing Products at Scale - Shopify

By: Jeet Mehta and Kathy Ge With over 1M business owners now on Shopify, there are billions of products being created and sold across the platform. Just like those business owners, the products that they sell are extremely diverse! Even when selling similar products, they tend to describe products very differently. One may describe their sock product as a “woolen long sock,” whereas another may ha

SEShopify Engineering·April 30, 2020Distributed
Distributed — The DynamoDB Book: An Interview With Alex DeBrie On His New Book - High Scalability -

The DynamoDB Book: An Interview With Alex DeBrie On His New Book - High Scalability -

You know nothing about DynamoDB. At least that’s what I realized the first time I heard Rick Houlihan give his now infamous talk at AWS re:Invent 2018 on Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB. In that talk Rick revealed for the first time the inner arcana of single-table design. Minds were blown. Weaknesses were revealed. Futures were changed. As a mere novice in the way

HSHigh Scalability·April 27, 2020Distributed
Distributed — One Team at Uber is Moving from Microservices to Macroservices - High Scalability -

One Team at Uber is Moving from Microservices to Macroservices - High Scalability -

There may be an undiscovered tribe deep in some jungle somewhere that hasn’t made up their mind on microservices, but I doubt it. People love microservices or love to hate microservices. There’s not much in between. So it means something when even a team at a company like Uber announces a change away from microservices to something else. What? Macroservices. But we’ll get to that. Think what you w

HSHigh Scalability·April 8, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For March 27th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For March 27th, 2020 - High Scalability -

Hey, it's HighScalability time! Awesome explanation of how to build a PID controller to fly a rocket! (BPS.space via Orbital Index) Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world. Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon

HSHigh Scalability·March 27, 2020Distributed
Distributed — The Working Group's White paper on Japan and covid-19

The Working Group's White paper on Japan and covid-19

Author’s note: This is a copy of the white paper of the Working Group, a group of four pseudonymous professionals in Tokyo. I, Patrick McKenzie, was the primary author. It was distributed quietly during the week of March 25th, 2020. I have written an essay describing how this document came to be and demonstrating its provenance .

PMPatrick McKenziePatrick McKenzie·March 26, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For March 13th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For March 13th, 2020 - High Scalability -

Hey, it's HighScalability time! The top 10,000 most spoken words in English represented by a point in hundreds of dimensions where the distance and direction between points encodes the relationship between words. (roadmaps) Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world. Know someone who w

HSHigh Scalability·March 13, 2020Distributed
Distributed — Scaling to exabytes and beyond

Scaling to exabytes and beyond

Years ago, we called Dropbox a “Magic Pocket” because it was designed to keep all your files in one convenient place. Dropbox has evolved from that simple beginning to become one of the most powerful and ubiquitous collaboration platforms in the world. And when our scale required building our own dedicated storage infrastructure, we named the project “Magic Pocket.” Two and a half years later, we’

AGAkhil Gupta·March 9, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For February 28th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For February 28th, 2020 - High Scalability -

Wake up! It's HighScalability time: The Smithsonian has million of pieces of delicious open access content. I ate up this 3D representation of the 1903 Wright Flyer. Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. That's how important you are to the fate of the world. Need to understand the cloud? Know someone who does? I wrote Explain the Cloud Lik

HSHigh Scalability·February 28, 2020Distributed
Distributed — Socratic vs. Euclidean Forms of API Documentation - High Scalability -

Socratic vs. Euclidean Forms of API Documentation - High Scalability -

I was emailing a service about their documentation and while their doc was good, about one particularly tricky concept they told me that once you use it for a while, that’s when you’ll understand it. In other words: you’ll only understand it after you understand it. I didn’t like that response. I want documentation that takes me from an unproductive newbie to a somewhat functioning journeyperson.

HSHigh Scalability·February 24, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For February 14th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For February 14th, 2020 - High Scalability -

Wake up! It's HighScalability time: Visualize the huge scale of Deep Time by identifying key reference points along the way. Do you like this sort of Stuff? Without your support on Patreon Stuff won't happen. I also wrote Explain the Cloud Like I'm 10 for everyone needing to understand the cloud (who doesn't?). On Amazon it has 93 mostly 5 star reviews (152 on Goodreads). Please be a real cloud he

HSHigh Scalability·February 14, 2020Distributed
Distributed — How To Empower Design Teams By Measuring Value — Smashing Magazine

How To Empower Design Teams By Measuring Value — Smashing Magazine

How do we know if we are truly impactful as a design team? Are we seen as a vehicle to deliver a solution that moves a needle? The business value of design has been proven at scale by the McKinsey Design Index. and the study shows the best design performers increased their revenues and shareholder returns at nearly twice the rate of their industry counterparts. It’s time to empower our design team

DCDave CunninghamDave Cunningham·January 24, 2020Distributed
Distributed — Stuff The Internet Says On Scalability For January 10th, 2020 - High Scalability -

Stuff The Internet Says On Scalability For January 10th, 2020 - High Scalability -

Wake up! It's HighScalability time: Forrest Brazeal warns you not to spend your limited innovation credits building bespoke, complex systems that provide no direct value to your business, instead position yourself at the top of what heI calls the Wisdom/Cleverness Curve Do you like this sort of Stuff? Your support on Patreon is appreciated more than you can know. I also wrote Explain the Cloud Lik

HSHigh Scalability·January 10, 2020Distributed
Distributed — The Split Personality Of Brutalist Web Development — Smashing Magazine

The Split Personality Of Brutalist Web Development — Smashing Magazine

No frills, or flashing neon frills with sprinklers attached? ‘Brutalist’ websites have flourished in recent years, but their guiding philosophy remains unclear. Brutalist web design has grown so quickly that there does not seem to be a clear consensus on what the style actually is. To some it means practicality, to others audacity. Love it or hate it, brutalist architecture celebrates rawness.

FOFrederick O BrienFrederick O Brien·January 8, 2020Distributed