Numbers That Tell the Scale Story

Every week brings a fresh set of numbers that put the industry's scale, speed, and reach into perspective. From database throughput records to the physical limits of long-haul communication, these figures are a snapshot of where distributed systems and digital services stand today.

Highlights From the Week

  • 54 million: peak transactions per second on a single DynamoDB table during Amazon Prime Day.
  • 200ms: the time saved per page load by enabling HTTP/2 server push; sending Link headers can make applications load 25–50% faster.
  • 30 miles: the distance over which quantum entanglement was successfully demonstrated through optical fiber, a step toward a super-secure internet.
  • 2 billion: the number of WhatsApp users, up from 1.5 billion just two years prior.
  • 1400%: the surge in job postings for augmented and virtual reality roles, compared to a 9% increase for blockchain positions.
  • 29: the average lines of code produced per day by @antirez over a decade of work on Redis.
  • 70%: Amazon Echo's share of the US smart speaker market.
  • 85%: the reduction in app storage requirements achieved by streaming software directly to phones.
  • 30%: of users now pay for at least one video game subscription service.
  • 34 hours: the round-trip communication time between Earth and Voyager 1, operating 11.5 billion miles away.
  • 135,000: pneumatic tube messages processed daily by Sears at its peak, a century-old predecessor to modern message queues.
  • 450 kilometers: of pneumatic tubes running beneath Paris streets in 1945, with canisters traveling at roughly 22 MPH and self-routing based on destination addresses.
  • 4.3 billion: records exposed in US data breaches over the last 15 years.
  • $1.5 billion: Alibaba Cloud's quarterly revenue, reflecting 62% year-over-year growth and a solid fourth-place position in the global cloud market.

California remains the top state for tech employment, with Amazon taking the #1 spot as the largest US tech employer. The state shows no signs of ceding its dominance in the sector. On the device side, Amazon's Echo family controls 70% of the US smart speaker installed base, reinforcing the company's grip on the voice assistant market. In the cloud infrastructure race, Alibaba's continued expansion demonstrates that the challengers are closing the gap with meaningful revenue growth. Meanwhile, the workforce is shifting dramatically toward AR/VR skills, a demand curve that far outpaces the more modest growth in blockchain-related roles.

The storage efficiency gain from app streaming is a reminder that the network itself can be a powerful ally in reducing on-device resource consumption. Similarly, the HTTP/2 server push figures suggest that relatively small protocol-level changes can deliver outsized performance improvements for web applications. And for sheer engineering nostalgia, the pneumatic tube networks of 20th-century Paris and Sears' message-routing operation show that the core challenges of store-and-forward communication and addressable routing are hardly new — only the substrate has changed.

The Scalability Roundup: February 14, 2020

IP: A 40-Year Lesson in Radical Minimalism

Ali Ghodsi reflects on the endurance of the Internet Protocol, which has carried the internet from nothing to roughly 15 billion connected devices over four decades—through the fastest period of technological change in history. His thread draws out a core lesson: the economic flywheel IP created. More bandwidth providers attract more developers building applications, which draw more users who, through those apps, demand more bandwidth. The enabling factor has been IP's radical minimalism and generality, surviving even as the underlying hardware has improved by a factor of 1,000,000x in moving bits from A to B.

Data: The Cost of Memory and Storage Realities

A conversation on Hacker News digs into the real-world mechanics of moving large datasets. One commenter notes that with data in S3, you can typically push ~20-40MB/core/sec on most instances. On an x1e.32xlarge, which benchmarks at about 2.7GB/sec according to Daniel Vassalo's S3 benchmark, fetching a 4TB database might take ~30 minutes. The bill: roughly $2 for compute, plus fractions of pennies for the few hundred S3 requests. Another voice points out the baseline on a small machine with a single SSD: at least 10-20 minutes for a pass over 1TB of data, even with perfect pipelining.

A related anecdote comes from a migration off HP-UX. A team moved a 600GB Oracle test database to a new HP DL980 server with 1TB of memory. The first test query run took ~45 minutes—already several hours faster than the old system. After some tuning, later runs completed in about 5 minutes. The catch: manually dropping buffers and cache sent the time straight back to ~45 minutes. The lesson, as one commenter put it, is that for highly tuned and specific workloads, running MySQL on EC2 might make more sense than RDS, which hides the knobs you'd need in that scenario.

On Engineering Culture and Architecture Choices

A former Googler offers a correction to the usual narrative around Android's launch. In 2007, Google's existential threat wasn't Apple—it was the US mobile carriers. Google wanted leverage to bring its products to feature phones and had no intention of releasing Android anytime soon. Then Apple shipped the iPhone. The reaction inside Google was immediate: "oh shit this is WAY better than the janky, blackberry-esque prototypes we have in development." The scramble produced the HTC G1/Dream on T-Mobile. Even with a Nokia N95 in hand, the Dream was laughably bad, but it did offer better mobile web browsing and application development—and, in hindsight, Android remains largely a reactive product with clumsy v1 holdovers.

Elsewhere, the debate over microservices continues. One commenter argues they are for companies with 500+ engineers, enabling strong ownership boundaries and deployment cadence. With 20 people, "you're doing it wrong." Another thread contrasts healthcheck strategies: shallow checks only confirm the service is up, while deep checks validate connectivity to S3, Postgres, Redis, Elasticsearch, and other dependencies. And on the topic of consistency, Nick Craver pleads against dynamic schemas in JSON payloads—making it an object for one item and an array for several breaks platforms and kills efficiency.

Storage and the Road Ahead

Enrico Signoretti predicts the hard disk will disappear from the small datacenter entirely—no more in homes, small businesses, or medium enterprises. Those organizations will rely on flash and the cloud. Future high-capacity drives (18TB, 20TB, moving toward 50TB) will be built for hyperscalers like Facebook and eBay, and public cloud providers such as AWS and Azure.

On the processing side, UPMEM claims first silicon-based benchmarks for processor-in-memory technology. A Xeon server with UPMEM's PIM DIMMs can perform eleven times as many five-word string searches through 128GB of DRAM as the Xeon alone, at one sixth the energy. Optimized parallel algorithms push that to up to 35x faster. The same system is said to sequence a genome ten times as fast, again at a sixth of the energy.

Protocols, Languages, and the Shape of the Future

Simon Peyton Jones reflects on how invisible software's complexity is: "99.9% of the population has no visceral... gut feel for just how complicated, remarkable and fragile our software infrastructure is." All his intellectual output, including GHC, would fit on a USB stick—a reminder that the invisible is no less intricate.

From Netflix, Justin Ryan describes the move from REST to gRPC as a huge win. The protocol's schema definition brings productivity gains; generating a single, thin client ended the habit of building heavyweight REST clients. Meanwhile, a commenter distills functional programming to a single principle: "it's about function composition. If you haven't picked up on this notion, then you haven't fully understood FP." Or, as another puts it, the worst part of a system is rarely the implementation—"if the protocol says that 'this happens, then this has to happen,'" the natural sequential part is the bottleneck you have to break apart.

Geoff Huston notes that subsea cable evolution is far from over. Optical transmission capability has increased ~100x per decade for three decades, but sustaining that pace will require considerable innovation. In quantum computing, Bill Fefferman sees a critical juncture: the Google/UCSB result shows a 53-qubit machine competing with a $0.5B classical supercomputer—even if simulation took 2.5 days rather than 10,000 years. A vibrant industry is emerging, with Google pushing experimental prototypes and IBM building a user base through cloud-deployed machines.

Quote Board

  • @jimmy_wales: "We [Wikipedia] already store data. In a database [as opposed to blockchain]. It works well."
  • @QuinnyPig: "Myth: Your AWS bill is a function of how many customers you have. Fact: Your AWS bill is a function of how many engineers you have."
  • crimsonalucard: "Functional programming is about function composition."
  • mantap: On SpaceX vs. Boeing: "SpaceX was supposed to be the risky new startup and Boeing was supposed to be the safe backup option. How times have changed."
  • jnwatson: "100% remote is far easier than 10% remote, because all of your processes must evolve around that."
  • Ryan Zezeski: On why projects fail beyond the code: "as any software veteran knows, projects often don't survive the whims of management."

Branch Prediction and the Modern CPU

Modern processors have abandoned the simple in-order execution of the past. Instead, they fetch a window of roughly 500 instructions, build a dependency graph between them, and execute the independent micro-graphs deeply out of order. Heavy bookkeeping tables ensure the results appear to complete in the original program order, but speed demands finding all available parallelism in that window. While there is a persistent myth that computers should be simple and clean, the market for simple, slow machines is effectively zero.

Today's microprocessors find about 10x parallelism within that window. This relies on branch prediction that succeeds in the high 90 percent range. Twenty years ago, the predictor simply recorded the last direction a branch took and guessed the same again, achieving roughly 85 percent accuracy. Modern predictors resemble a neural network, performing deep pattern recognition on program flows, often multiple ways simultaneously, before selecting a direction. This effectively embeds a tiny supercomputer in the CPU solely to guess branch directions. The jump from 85 percent to 99 percent accuracy required an increase in prediction state from about 1,000 bits to tens of megabits, as the window of instructions under analysis grew from 50 to 500.

Despite this deeply non-deterministic execution flow, computers still return a deterministic answer. Running a program 100 times will never execute it the same way twice, yet it produces the same result every time.

Innovation, Diminishing Returns, and Reinvention

Progress in computer architecture is not a single curve but a stack of thousands of innovations, each with its own trajectory. Transistor shrinkage is one visible piece, but under the hood, each technique—from materials science to circuit design—has its own diminishing return curve. An expert on one curve who sees its plateau may declare the field finished, while others are elsewhere pushing a different set of innovations, yielding the overall exponential progress we call Moore's Law.

Given this, a project first ascends and then exhibits diminishing returns. Reaching the next level requires starting fresh, even if the new project begins below the old one's plateau. This tension pits short-term disaster (fear of changing what works) against long-term disaster (stagnation), with the former dominating quarter-by-quarter business objectives.

The scaling horizon remains large. A modern transistor measures about 1,000 x 1,000 x 1,000 atoms. Quantum effects start to appear at 2-10 atoms, suggesting a possible transistor just 10 x 10 x 10 atoms—a million-fold reduction in volume. Techniques already exist to place atoms in single atomic layers, but manufacturing constraints dominate: placing atoms one at a time is far too slow for the 10^23 atoms needed for a computer. The innovation stack required to bridge this gap remains wide and deep.

NoSQL: Tuning the Model to the Access Pattern

The fundamental difference between relational and NoSQL databases comes down to where the work happens. Relational databases normalize data across multiple tables, then burn significant CPU on every request to re-format and join that data on the fly to satisfy ad hoc SQL queries. NoSQL eliminates that overhead by denormalizing data to fit the application's access patterns, allowing simple queries to satisfy requests without joins. A join, after all, is essentially a grouping of objects—the same result can be achieved with an index scan in a NoSQL table at far lower cost.

The most common NoSQL mistake is carrying over multi-table relational designs. Multi-table designs are never efficient in NoSQL, at any scale. The primary design question is not about individual components or features but about the known, repeated access patterns: roughly 90 percent of applications execute a limited set of queries regularly. The first step is identifying those queries and modeling the table specifically to serve them.

Denormalization does not always mean copying data. One customer's quote system is the canonical example: instead of storing the entire revised quote each time, they stored the first version and then appended deltas for every change. The client could replay the deltas to show the current or any historical version. This took their write capacity provisioning from 1,000 WCUs down to 50—a 95 percent reduction. The lesson is clear: don't store data you don't need. Similarly, avoid building gigantic objects for low-value access patterns; serving a customer a few slots to choose from beats delivering a monolithic 16 MB array of all available appointments. The proxy for effective design is simple: push triage logic to the client, preload the two or three likely follow-up requests to save round trips, and stay aware you pay for the amount of data you read—not the number of items.

The same logic applies to architecture. Vendor lock-in is not about the database software but about the data itself. Once ten terabytes sit in one system, moving remains a time-consuming operation whether on MongoDB, Cassandra, DynamoDB, or Cosmos DB. At the lowest common denominator, all are equivalent in their basic query—select * where X equals—and choice depends on the velocity and access pattern of the workload. For OLTP applications in particular, the economics are stark: data center utilization for enterprise applications hovers around 12 percent, meaning 88 percent of infrastructure spend is wasted. One customer deprecated a three-instance MongoDB cluster at $500/month for DynamoDB at $50/month, a 24-hour porting effort included.

Kafka's Speed: Sequential I/O and Client-Side Work

Apache Kafka achieves high throughput, trading some latency and jitter, while retaining durability, strict record order, and at-least-once delivery. The core insight is that disks are not inherently slow—random I/O on a 7,200 RPM SATA disk is three to four orders of magnitude slower than sequential I/O. Kafka exploits this by using a segmented, append-only log that confines itself to sequential reads and writes. Records are then batched on both the producer and the consumer before traversing the network, amortizing round-trip overhead and improving bandwidth efficiency.

Batch compression multiplies this effect, particularly with text formats. JSON, for instance, often compresses at least 5x-7x. Because batching is a client-side operation, compression offloads work from brokers, improving both network bandwidth and the brokers' disk I/O utilization. Consumers remain cheap because they do not mutate logs. Kafka tracks offsets per consumer group in its own internal topic and compacts it to retain only the latest known offsets.

Kafka's most consequential choice is that it does not fsync a write before acknowledging; a record need only reach the OS's I/O buffer to be accepted. Combined with heavy use of the page cache, this makes Kafka behave like an in-memory queue, albeit one backed by disk and bounded by available buffers.

The design pushes significant work onto clients. Producers stage records in accumulators, hash keys to partition indexes, checksum and compress batches, and stay aware of cluster metadata to route writes directly to partition masters. Consumers can similarly fetch from geographically local replicas to cut latency. Kafka further leans on a binary message format shared by producer and broker so compressed chunks flow unmodified end to end. On Linux and UNIX systems, Kafka uses Java's NIO transferTo() method to move data between channels without involving the application as an intermediary—the platform's zero-copy mechanism.

This common emphasis on channels, native buffers, and page cache yields a meaningful side benefit: less pressure on the garbage collector. With reduced GC pauses, brokers are less likely to introduce jitter in propagation delay (such pauses can otherwise be extended across an end-to-end chain, and the end-to-end record propagation delay).

Scale comes from concurrency baked into partitioning and consumer groups, which load balance partitions across members in the group. By comparison, a traditional MQ like RabbitMQ assigns messages to concurrent consumers in round-robin fashion, forfeiting any global ordering guarantee in the process.

Memory Bandwidth vs. Latency in Practice

The actual numbers for modern 2020 silicon circuits are striking at the hardware level, even as they highlight how vague many programmer heuristics remain. Per core, L1 latencies sit around 1 ns, L2 around 2.5 ns, and L3 around 10 ns, with RAM at a 50 ns roundtrip. Bandwidths show symmetrical widening: a full system sees about 210 GB/s for L1, 80 GB/s for L2, and 60 GB/s for L3 per core in terms of raw bandwidth upper limits, while RAM bandwidth tops out at roughly 45 GB/s per socket—a common roadblock that emphasizes careful attention to access patterns since a single random access can't realize that. That upper limited view only holds for sequential throughput; using those later figures directly as guidelines on every use case suggests typical real-world estimates of 5 GB/s with a lower bound near 1 GB/s. This contrast means sequential access to RAM’s speed is deceptive: it's fast only because whole cache lines move hidden behind the interface.

These values mean random access into the cache is comparable to sequential RAM performance. Random access that repeatedly strays into main memory is catastrophically slow, under 1 GB/s for int32. Distinct findings follow: pointer chasing runs 10 to 20 times slower than any list-savvy representation.".

Serverless: Microfunctions vs. Microservices

Implicit in every serverless decision is the question: are you deploying a microfunction or a microservice? Serverless frameworks allow rapid application development, but running the same code elsewhere is harder, and diagnosing problems is limited because the platform holds many levers beyond your reach. Adopt clean architecture principles, abstracting core use cases from the interfaces—the surrounding input/output. Otherwise the application code becomes portable code waiting for a problem to happen.

Modeling an entire domain with multiple aggregates and entities into one Lambda is a mistake. A more nuanced handling suggests an ECS cluster with EC2 launch type starting place for monitoring usage in production, then tuning and finally deploying to Fargate. Lambda and Fargate expose comparable operational risks. In the clearest example, you should deploy the domain as a single dockerized microservice—not break up six supporting Java methods into six separate Lambdas.

Practical limits with the model cover the need to maintain a client-server separation where statelessness meets real resource management. Socket connections, and HTTP/JDBC pooled resources cannot simply be created and destroyed with each function. The Lambda instance lifecycle forces containment if needed. In both ECS and Lambda, resource management eventually lands on developers.

Open Source Finds

olric is a distributed, eventually consistent, in-memory key/value data store and cache. It works both as an embedded Go library and a language-independent service, allowing you to spin up a fast, scalable shared pool of RAM across a cluster of machines.

The Memory Pool System Kit from Ravenbrook provides the complete sources for using, modifying, and adapting the MPS memory management system. The accompanying documentation gives a brief overview and pointers to more detailed information.

Publications and Research

Kubernetes for Full-Stack Developers

DigitalOcean's book for developers covers Kubernetes core concepts, modernizing applications for containers, containerizing applications, deploying to Kubernetes, and managing cluster operations.

Canopus: A Scalable Consensus Protocol

Canopus is designed for high throughput and scalability relative to the number of participants. It gains throughput primarily through batching and achieves scalability by parallelizing communication along a virtual overlay leaf-only tree (LOT). The protocol explicitly trades off latency and fault tolerance in favor of throughput.

Automated Performance Regression Diagnosis

Finding the source of performance bugs introduced by software updates is a manual, time-consuming process. Researchers at Texas A&M University and Intel Labs describe a fully automated approach in their paper on zero-positive learning.

Modern desktops and servers expose hundreds of performance counters, making manual monitoring and anomaly detection impractical. The team's deep learning algorithm monitors many counters simultaneously by reducing the dimensionality of the data—analogous to compressing a high-resolution image by changing its format. The algorithm then looks for patterns in the lower-dimensional data that deviate from normal behavior. This turnkey method identifies performance bugs in hours rather than days.

AI Shortcuts for Scientific Simulations

Simple AI shortcuts can speed up simulations of everything from galaxies to atoms by billions of times, as reported in Science Magazine. The technique, called Deep Emulator Network Search (DENSE), relies on a neural architecture search developed by Stanford's Melody Guan. It randomly inserts computational layers between a network's input and output, then tests and trains the resulting configurations with limited data. Layers that improve performance are more likely to persist in future iterations. Muhammad Kasim, a physicist at Oxford who led the study posted on arXiv in January, built on Guan's work because it balanced accuracy with efficiency.