Scale Notes from Around the Web

Quantinuum and other system builders are pushing the boundaries of what's possible in computing, but sometimes it's the numbers from everyday internet infrastructure that put scale in perspective. Here is the latest collection of noteworthy metrics from across the industry.

Data Center and Infrastructure

Twitter's MetricsDB

Twitter reports ingesting 5 billion metrics per minute, amounting to roughly 1.5 petabytes of data. The company has documented its approach to managing this firehose of operational telemetry.

DDoS Attack Landscape

AWS mitigated a 2.3 Tbps DDoS attack, one of the largest ever recorded, with UDP traffic again being the primary vector. For contrast, Cloudflare notes that the vast majority of attacks it handles are much smaller—92% of its Q1 2020 mitigations were under 10 Gbps, and nearly half were under 500 Mbps.

Hard Drive Longevity

Backblaze reports an annualized failure rate of 1.07% across its fleet of 129,959 drives, the lowest mark the company has seen.

Computing and Performance Benchmarks

Fugaku Takes the Crown

Japan's new ARM-based supercomputer, Fugaku, performs 2.8x more calculations per second than IBM's now-second-place system. The machine carried a reported price tag of $1 billion.

The Cost of Data Movement

Rice University researchers highlight a significant bottleneck in AI workloads: 90% of electricity consumption goes toward moving data between memory and the processor. Accessing DRAM can cost roughly 200 times more energy than performing a computation itself.

Optical Chip Speed Record

Australian researchers achieved 44.2 Terabits per second using a micro-comb, an optical chip that replaces 80 separate infrared lasers for carrying communication signals.

Business and Market Signals

Cloud Economics

Estimates suggest that AWS accounted for 50% to 80% of Amazon's profits, though some analysts debate the precise figure based on differing accounting methods.

Gaming and Apps

Pokémon Go generated $1.4 billion in revenue last year, ranking sixth among free-to-play games, with Fortnite taking the top spot. Weekly player spending in the game jumped 70% between March 9 and March 16 amid the coronavirus outbreak, hitting $23 million.

Minecraft reports 126 million monthly players, with total sales topping 200 million copies.

Hardware Market Declines

Smartphone sales fell 20% in Q1 2020, according to Gartner. In another sign of industry change, Olympus announced it is exiting the camera business after 84 years.

Platform Behavior and Content

Algorithmic Influence

A Media Matters report claims that 64% of all extremist group joins stem from Facebook's recommendation tools. Separately, researchers found that nearly 50% of accounts tweeting about coronavirus are likely bots.

An analysis of 95% of Android apps found they contain dark patterns effective at manipulating user behavior.

Traffic and Crawlers

One developer documented how a Facebook crawler was hitting his small website with 7 million requests per day, peaking at 300 requests per second, until he blocked the user-agent via Cloudflare rules.

Podcast and Social Media Numbers

The 10 world-leading podcast publishers generated 900 million streams and downloads in a single month, with iHeartRadio leading at 216.25 million. TikTok expects US revenue of $500 million this year, amid a private-market valuation of $100 billion for its owner.

YouTube turned 15 years old this year. Meanwhile, IT job losses since March total 117,000.

Developer Preferences and Mobility

Language Demand

In a survey of developers, 86% expressed a desire to use Rust, followed by TypeScript at 67.1% and Python at 66.7%.

Google Maps data for California showed people stayed home 9% more compared to the baseline period from April 25 through June 6. Retail and recreation visits were down 37%, while workplace attendance fell 18%.

Quotable Bits From the Scalability Discussion

ARM's economics get a shout-out from James Hamilton: the sheer volume of ARM processors shipped everywhere drives the low-cost economics and R&D investment needed for serious server silicon. Kevin Kelly adds a softer but no less functional note: enthusiasm is worth 25 IQ points.

Twitter's engineering blog reports that a custom storage backend cut costs by a factor of 10 and latency by a factor of five versus a traditional key-value store, which in turn enables features requiring higher scale. Elsewhere, the conversation turns to AWS — @QuinnyPig calls non-competes bad for everyone, while @timallenwagner imagines the CDK not as a dev tool but as an embeddable layer for build and control capabilities baked into every AWS-hosted application. On serverless, @rts_rob passes along AWS's reasoning for building Lambda on traditional VMs: containerization is fast and resource-efficient but doesn't give strong isolation between customers. @ben11kehoe pushes back on the complexity narrative: serverless moves complexity from code into infrastructure rather than creating it from nothing. A Hacker News commenter counters that stateless serverless forces a data-shipping architecture unwieldy for stateful workloads, and pans Lambda + EFS as shared mutable state with brittle semantics; someone else points to Athena user-defined functions as a better fit for map-reduce-style jobs than S3- or EFS-shuttling Lambdas.

AWS veteran Tim Wagner reflects on Lambda's early days: the team didn't anticipate how fast it would grow or that it would need a development framework and client-side support of its own, admitting a bit of "AWS-itis" in assuming a great service would be consumed however users needed. On security, @alexstamos notes none of the major video-conferencing players offer end-to-end encryption by default; WebEx has an enterprise-only option requiring users to run their own PKI.

Performance and latency debates pop up repeatedly. @hichaelmart points out Node.js has fallen behind in the TechEmpower benchmarks, while es4x (JavaScript on the JVM) is 5x faster for Postgres updates. A Hacker News commenter frames latency as the ultimate IO resource: a direct method call in warm L1 returns within nanoseconds to microseconds, while politely asking a Lambda over the internet takes milliseconds to seconds — six orders of magnitude or more. Another commenter's S3 story is a reminder that heavy IO, not raw compute, is often the bottleneck: pulling a lot of images peaked at 800 Mbit/s downstream before the bucket went offline entirely.

On retries and backoff, Shubheksha explains that combining exponential backoff with jitter spreads retries evenly across time, giving an exhausted node room to finish in-flight requests instead of being hammered by synchronized retries. @colmmacc draw an analogy between cryptographic seeds and Sequoia seeds, both of which generate output vastly larger than themselves. In fault-tolerance philosophy, Ian M. Banks' words surface: "Strength in depth; redundancy; over-design." Grady Booch's line gets repeated too: architecture is whatever hurts when you get it wrong.

On languages, Joab Jackson writes that the industry is moving toward Rust, citing usage at AWS (Lambda, some of EC2), Facebook, Apple, Google, Dropbox, and Cloudflare. A commenter wishes for better hardware-accelerated bounds checking — cacheline faults, poison words, bounded MOV instructions, hardware ASLR, auto-encrypted cachelines, and per-process encrypted memory to blunt Spectre-style attacks.

Several threads touch on scale economics and workloads. qeternity pays around $2k for an analytic cluster with hundreds of cores, over 1TB of RAM, and many TB of NVMe — and even idle, it's an order of magnitude cheaper than alternatives. joshuaellinger reports a large enterprise moving all its compute to Spark on Azure to decommission expensive Teradata instances: a big step forward for batch, dismal for interactive, driven largely by price. @ID_AA_Carmack does the math on reading: one billion people reading for an hour at under 250 words per minute would only generate about 75 TB of traffic, under $10k in costs.

Recurring themes include the gap between promise and practice in AI, with @cote citing that seven out of ten companies said their AI projects had little impact so far, and two-fifths with significant investments reporting no benefits yet. @ID_AA_Carmack gets the last word on the state of things: "I can send an IP packet to Europe faster than I can send a pixel to the screen."

Basecamp’s HEY Stack, From Rails to Kubernetes

Basecamp’s new email service HEY has attracted plenty of attention, not least for its technology choices. The company went with a deliberately unfashionable stack: vanilla Ruby on Rails on the backend, running on edge (the latest version off git), with Stimulus, Turbolinks, and Trix on the front end, plus some unreleased “NEW MAGIC.” Data lives in MySQL (Aurora, with Vitess for sharding); Redis handles short-lived data and caching; ElasticSearch does indexing. The whole thing runs on AWS via EKS, with an ALB for load balancing, Resque for background jobs, Postfix and ActionMailbox for mail delivery, and a Go-based caching image proxy called imageproxy.

The architecture follows what DHH calls The Majestic Monolith—an integrated system that collapses unnecessary conceptual models and avoids distributing the system unless truly needed. Development took two years. HEY is already handling 30,000 web requests per minute, runs 91% of its app instances as spot instances, and is 99% Kubernetes. A second region is in standby and will go active/active soon. Users are multi-tenant in the same DB, and MySQL is not running on Kubernetes—it’s Aurora. HTML is delivered over websocket.

@t3rabytes credits Kubernetes for more than lock-in avoidance: “we have pretended over deploying, logging, monitoring, cluster management and access, Kubernetes actually has a CLI that’s useful, and we’ve found Kubernetes to be (gasp) less of a black box than ECS.” Cloud becomes nice, he notes, when you plan for X users and the world opens up to 4X—scaling looks like a few YAML updates and waiting minutes. Spot instances have had termination quirks, but the push to use them as much as possible has paid off.

DHH is candid about tradeoffs: on-prem is “MASSIVELY cheaper than cloud for compute and big dbs over the long run,” but HEY would have been “toast launching in Orem.” GCP “nearly burnt Basecamp to the ground,” he says. There’s a plan to be “big tech free on infrastructure in five years,” but no suitable non-big tech clouds exist yet. On the front end, DHH’s summary: “React is so 2019. HTML + minimum JS is 2020.”

One commentator notes Turbolinks generates everything server-side and calculates the delta between existing and incoming templates, injecting only changes—meaning no client-side state to care about. A deeper dive is available in A few sneak peeks into Hey.com technology.

Khan Academy’s 2.5x Traffic Week

In April, Khan Academy served 30 million learners, handling 2.5x normal traffic in a week. The organization says it scaled readily because of architecture and careful use of external services. The backbone: Google Cloud (AppEngine, Datastore, Memcache), Fastly CDN, and YouTube for video offload. App Engine’s fully managed environment scales with virtually no effort; Datastore scales out storage and access capacity automatically. Fastly caches static data, minimizing server trips and keeping hosting costs from growing linearly.

The team caches common queries, user preferences, and session data liberally in Memcache. Early slowdowns traced to deploys; Google increased Memcache capacity and within a week normal continuous deployment resumed. Engineer dangoor adds color: a lot of the site is dynamic—tracking progress, dashboards for teachers and parents, integrations with test providers—so PostgreSQL for everything would have meant scrambling. App Engine costs more than raw Compute Engine, but the tradeoff is less engineering/ops time and less after-hours work. A Python library of function decorators generates cache keys based on function names and parameters for different data access patterns.

Scaling Rules of Thumb

Six rules of thumb from architect Ian Gorton for scaling software systems:

  • Cost and scalability are indelibly linked. Design decisions for scalability inevitably affect deployment costs.
  • Your system has a bottleneck. Somewhere! Any shared resource can limit capacity as you grow microservices. Guard against sudden crashes and deploy capacity quickly.
  • Slow services are more evil than failed services. An overwhelmed service causes cascading failure—Circuit Breakers and Bulkheads are the safeguards.
  • The data tier is the hardest to scale. Confront this as infrequently as possible.
  • Cache! Cache! And Cache Some More! Handling reads from cache buys extra database capacity.
  • Monitoring is fundamental. You need to understand relationships—database write performance under concurrency, circuit breaker trips, new instances spawned, queue depths.

There’s also a real-world parallel: COVID “cohorts” for school reopening are the same idea as the bulkhead pattern—decreasing the blast radius by isolating groups.

Graviton2 Instances and the Serverless Frontier

AWS’s James Hamilton announced new M6g, C6g, and R6g instances powered by Graviton2. The chip is 64-bit Neoverse with ~30B transistors on 7nm, offering 7x better performance over v1, 4x core count, 5x faster memory, and 2x cache. The instances deliver up to 40% better performance over comparable x86. PostgreSQL on M6g is 46% faster with 56% better response time than M5; MySQL is 28% faster with 47% better response time. Honycomb.io ran 30% fewer instances on C5-equivalent hardware, each 10% cheaper, for 47% all-in savings.

Tim Wagner’s interview on Lambda’s past and future covers disaster recovery support, edge-first operation, better CI/CD integration, built-in idempotency and circuit breaker mechanisms, stateful application support, millisecond billing granularity, EFS integration, networking, and efficient Lambda-to-Lambda communication for distributed algorithms. He got his wish on EFS.

Tom Wallace’s meta-programming Lambda talk explores generating custom code from within a Lambda—even redeploying Lambda code from within Lambda. Bespoke Lambda functions per user cost nothing until they run, so custom versions implementing per-customer configs make sense. He also describes a master Lambda starting hundreds of other Lambdas for map-reduce over IoT data; Step Functions don’t ramp up parallelism fast enough. Lambda can spin up thousands quickly, but with retry logic needed for large starts. Queries running in a separate account prevent starving other users of the Lambda pool. A spot scheduler for on-demand work—executing whenever, optimizing for cost or green energy—would be a good feature.

Serverless Economics and Language Choice

Bobby Allen of CloudGenera is blunt about cloud repatriation: building your own datacenter can mean missing customer conversations to save a few bucks on AWS. There’s a tipping point where IaaS/PaaS becomes cheaper to run yourself—PaaS can be 10x IaaS. His warning: AWS runs loss leaders on the low end, and trusting them there leads to drinking the Kool-Aid at the high end.

In the serverless supercomputer vein, embarrassingly parallel compute treats Lambda as “unlimited on-demand compute threads.” Jobs spin up in as many workers as needed. One example: 3.5 billion webpages, or 198TB of data, processed in ~3 hours. Lift-and-shift is no cheaper in the cloud, but converting to serverless can save up to 90%.

Bearer.sh’s experience monitoring 30k API calls/min shows what happens when serverless costs bite. Hosted on AWS Fargate, autoscaling after 4000 req/min, the invoice came due. Moving logs from CloudWatch to Kinesis Firehose dropped daily costs to 0.6%. But GC pauses breached 4 seconds—up to 400 pauses per minute—thanks to a memory leak in the AWS Javascript SDK. Moving to Rust: latency dropped below 90ms even at peak (average below 40ms), memory usage from 1.5GB to 100MB, CPU from 150% load to 2.5%.

On the DAZN podcast, the team describes a “serverless first” approach—easy to use, operationally simpler, resilience free. But hybrid is normal. Spike loads like 1.2 million concurrent users before an event don’t fit Lambda’s 3,000 concurrency limit; an autoscaling group you can increase beforehand makes more sense. Active-active minimizes downtime; anything not in a critical path is active-passive. DynamoDB global tables get heavy use, with the caveat to process streams in only one region to avoid duplication.

Storage First, Pre-render, and Other Serverless Chats Takeaways

  • Storage first. Get data stored before processing so it isn’t lost in buggy pipelines; events from DynamoDB or S3 drive the application state machine.
  • Pre-render first. Most pages can be computed once, globally shared and distributed—no server round-trip per page view.
  • Integration tests. Validate assumptions about how code runs within the larger environment; unit/functional tests are for the code itself.
  • DynamoDB. Well-defined access patterns with limited, frequently executed queries—that’s the OLTP model.
  • NoSQL. Retail is evidence: Tesco crashed on Black Friday 2014 because it couldn’t handle online spikes, switched to NoSQL, and now can.
  • Serverless future. “What’s the alternative, really? More complexity, more configuration files, more weird container orchestration stuff?”

Tools for Distributed and Serverless Systems

Several open-source projects aim to close the gap between local development workflows and large-scale distributed execution:

  • StanfordSNR/gg — A framework and command-line toolkit that runs everyday tasks, such as compilation, unit tests, video encoding, or object recognition, across thousands of parallel threads on a cloud functions service.
  • lsds/faasm — A high-performance stateful serverless runtime. It provides multi-tenant isolation while allowing functions to share memory regions, which offers low-latency concurrent data access and global synchronization for large-scale parallelism.
  • hydro-project/anna — A low-latency, autoscaling key-value store from UC Berkeley's RISE Lab. Its core design goal is avoiding expensive locking and lock-free atomic instructions, which have recently proven to be extremely inefficient.
  • stanford-mast/pocket — A storage system built for ephemeral data sharing. It delivers fast, distributed, elastic storage for data with low durability requirements, and exposes a serverless abstraction so users do not manage storage servers manually. A primary use case is intermediate data exchange between execution stages in serverless analytics.
  • hse-project/hse — An embeddable key-value store designed for SSDs based on NAND flash or persistent memory. It optimizes performance and endurance by orchestrating data placement across DRAM and multiple classes of solid-state storage.

Research: Clocks, History, and Isolated Functions

The latest batch of publications features work on distributed timing, high-resolution metrics, DNA-based media, and lightweight serverless isolation:

  • CIRCLLHIST — A fast, memory-efficient histogram implementation for summarizing large numbers of latency measurements. It suits IT infrastructure monitoring, offering nanosecond data insertion, full mergeability, and accurate quantile approximation with a-priori bounds on relative error.
  • HybridTime — A technical report introducing a hybrid of physical and logical clocks for globally consistent databases. Unlike Spanner, HybridTime does not wait out time error bounds for most transactions, so it works with common time-synchronization systems. Experimental comparisons in the same system show it can outperform Spanner-style commit-wait by up to an order of magnitude in latency.
  • Dynamic and scalable DNA-based information storage — The DORIS framework demonstrates how simple innovations shift the physical and encoding architectures of DNA storage. Using ss-dsDNA strands, it provides a physical handle for isothermal file access, increases theoretical information density by inhibiting non-specific binding, eliminates hard design problems for orthogonal address sequences, enables repeatable file access through in vitro transcription, controls relative strand abundance, and supports in-storage file operations.
  • From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers — The gg framework evaluation covers several latency-sensitive applications. In the best case, a distributed compiler outperformed the conventional tool icecc by 2–5× without a warm running cluster. In the worst case, gg stayed within 20% of the hand-tuned performance of the video-encoding tool ExCamera.
  • Faasm: Lightweight Isolation for Efficient Stateful Serverless Computing — The paper introduces Faaslets, an isolation abstraction that uses software-fault isolation via WebAssembly to isolate function memory while allowing sharing within the same address space. CPU and network isolation come from standard Linux cgroups, and a low-level POSIX interface handles networking, file system access, and dynamic loading. Restoring from snapshots cuts initialization times. Compared with standard containers, Faasm achieves a 2x speed-up with 10x less memory when training a machine learning model, and doubles throughput while reducing tail latency by 90% for inference serving.

Learning and Listening

For those interested in a hands-on education resource, Dive into Deep Learning presents an interactive deep learning book with code, math, and discussions, offering both NumPy/MXNet and PyTorch implementations.

Wrapping up this edition: entrancing live performance from Ólafur Arnalds — "Undan hulu" recorded at the Sydney Opera House, from the composer known for programming dueling pianos.