Stuff The Internet Says On Scalability For February 7th, 2020 - High Scalability -

This Week in Scale: Solar-Powered Sites, Satellite Internet, and the Numbers Behind Big Tech

Every week brings a fresh set of metrics that reveal how the internet and the systems supporting it are evolving. From sustainable web hosting experiments to the latest earnings reports from tech giants, these figures offer a snapshot of the current state of scalability and infrastructure.

The Green Web and Smarter Signals

One of the more interesting experiments this week involves a solar-powered website that achieved 95.26% uptime. The project demonstrates that a single kilowatt-hour of solar-generated electricity can serve nearly 50,000 unique visitors, suggesting a viable path toward more sustainable web hosting.

In the physical world, MIT researchers have developed a smart wall surface that functions as a "mirror or lens" for radio signals. This innovative approach can boost WiFi signal strength by 10x and double median channel capacity by focusing signals directly onto devices on either side of the wall.

Energy efficiency is also getting a boost from smart building technologies. A smart heating and cooling control system demonstrated 25% less energy usage, while industry analysts project that the smart building software and services market will generate over $2 billion in revenue by 2026.

Cloud and Platform Earnings Highlights

The cloud wars continue to intensify, with Alphabet reporting a $10 billion run rate for Google Cloud, representing 53.6% growth year-over-year. Microsoft Azure saw even stronger momentum with a 62% year-on-year revenue increase, while its Dynamics 365 cloud-based enterprise resource planning service jumped 42%.

YouTube remains a massive revenue driver for Alphabet, generating $4.7 billion in Q4 revenue. The platform handles over 1 billion hours of video watched daily and is up 36% from 2018 and 86% compared to 2017. According to journalist Nick Statt, YouTube is now 20% the size of Facebook, contributes 10% of all Google revenue, and is 6x bigger than Twitch.

Platform Metrics at Scale

Several platform-specific numbers caught our attention this week:

• 150 million Amazon Prime members worldwide, with the company's retail fulfillment network spanning 175 centers and employing over 250,000 full-time associates.
• Duolingo now stores 31 billion items to deliver lessons in over 80 languages, processing 24,000 read units and 3,300 write units per second for its 18 million monthly active users. The platform handles 6 billion exercises per month with just 2 people in DevOps.
• Tinder processes 2 billion daily swipes and hosts more than 30 billion matches globally across 190 countries.
• Snapchat grew to 218 million users, up 3.8%, posting $560.8 million in revenue (up 44% year-over-year) despite a $241 million loss.
• Amazon Prime membership numbers echo Costco's business model, where membership dues drive profitability.

Connectivity, Commerce, and Infrastructure

Looking at internet access from a different angle, researchers suggest that just 4 satellites could provide continuous global coverage at a fraction of current costs. This could be transformative for connecting underserved regions.

EBay's investment in machine learning-driven translation is paying off, with an 11% increase in commerce attributed to the improved functionality. This shows how AI investments can have direct impact on business outcomes.

In hardware news, the semiconductor industry had its worst year since the dot-com bubble burst, with 12% reduction in revenue for 2019 and memory revenue dropping 33%. Meanwhile, Bluetooth Low Energy continues its march toward ubiquity, with projections that 90% of Bluetooth devices will support the standard by 2023.

On the performance side, browser benchmarks show Chrome is now 30% slower, a significant regression worth noting for those building web applications.

Finally, two fascinating non-tech-scale data points: researchers say mapping a complete mouse brain wiring diagram would require 2 exabytes of storage, and 143 new geoglyphs were discovered on the Nasca Pampa. The military maps collected by King George III amount to some 3,000 documents now published in an archive.

Networking Notes: The Cost of Latency and Compute

Opinions on serverless and FaaS architectures remain sharply divided. Joe Hellerstein questions whether the current patterns and antipatterns for serverless are sound, pointing to measured limitations of step functions and function invocation that make pure stateless functional programming impractical. In his view, first-generation FaaS constraints effectively limit programming to coarse-grained workflow, undercutting the platform's potential. Others are less measured: DHH calls serverless the industry's next "bigger delusion" after the microservices recovery.

Traveloka reports that moving to a serverless setup cut spending by more than 90% compared to an EC2 server, eliminated server maintenance, and reduced front-end content latency. The trade-off, per Jonathan Blow on Rust, is about experimentation cost: when working on genuinely hard problems, he says, a language or platform that drives the cost of experimentation too high impairs his ability to get work done.

The question of where compute happens drew a strong prediction from Mark Lapedus: today's microelectronics are 80% cloud and 20% edge; in five years, that ratio reverses. Privacy of data is cited as the underlying driver.

Concrete Performance Numbers

A few genuinely interesting measurements surfaced this week:

• A test with fewer than 100 Starlink satellites in orbit delivered approximately 600 Mbps to an aircraft in flight.
• Rehan van der Merwe wrote 1 million records into DynamoDB in 60 seconds from a single Lambda, roughly 17K writes per second, in under 250 lines of code plus 70 lines of CloudFormation. Running several importer Lambdas concurrently reached 40K writes per second (2.4 million per minute).
• Diego Pacheco benchmarked three HTTP servers: Fastify at 6K requests per second with 14ms latency, Netty at 23K RPS with 4ms latency, and Actix at 53K RPS with 1ms latency.
• Google's Meena chatbot was trained for 30 full days on a full TPUv3 pod (2048 TPU cores), estimated at more than $1.4 million of compute time — over 100 petaflops of sustained compute.

One AWS engineer, testing M6g instances privately, said that while formal benchmarks can't be shared, workloads ran uniformly better on Graviton2 than on the equivalent Intel-based M5 configuration. The claim: no performance is sacrificed for price, and for scripting languages, Java, Go, or recompiled C/C++, these instances are a "no-brainer."

The Cost of Infrastructure

GDPR and CCPA are shaping engineering practice in a specific way, notes @rbranson: data retention is now being considered at the start of projects consistently, since building with finite retention from the outset is an easy box to check.

An anonymous enterprise GCP customer enumerates frustrations with Google Support: linking to generic documentation that doesn't solve the issue, insisting obvious bugs aren't bugs until a trivial reproduction case is provided, refusing to advise on beta products (despite half of GCP being in beta), and occasionally being confidently wrong about root cause. Their verdict: "Give me AWS support any day."

Google's bug bounty program, per @etherealmind, cost the company 0.000040625% of gross 2019 revenue — roughly the cost of 27 full-time employee roles at a $240K average loaded salary, making it cheap. But Google may be spending engineering dollars elsewhere: relegate its Intel NICs to QuickData Technology DMA while offloading network encryption/decryption from server processors, with no public word on whether GCP has deployed SmartNICs.

Hardware Projections

Ivo Bolsens of Xilinx predicts FPGAs will eventually outnumber CPU nodes at a ratio of perhaps 16 FPGAs to every one CPU, as acceleration outweighs general compute. Meanwhile, on the silicon process side, Mark Lapedus details the triple structure of a chip — transistor, contacts, and interconnects — with the middle-of-line layer connecting them. Scaling data at the leading edge: 7nm processes use a contacted poly pitch of 56-57nm with 40nm metal pitch; at 5nm, CPP drops to roughly 45-50nm with a 26nm metal pitch.

Backblaze notes higher-than-typical failure rates among some of its 12TB Seagate drives. And the U.S. Navy moved its largest ERP system — 72,000 users across 6 commands — to the AWS Cloud, ten months ahead of schedule.

Architectural Lessons and Limitations

Ayende Rahien argues that certain applications have a hard cap on expected load, and architects should respect it. "The more stuff you can not do, the better you are," he writes. Simpler architecture means faster time to market, rather than preparing for what he calls the Babies' Apocalypse.

Werner Vogels reflects on AWS's early conflation of identity and account: identity is a security component, while account is what you bill to. His lesson is that had they separated the two from the start, later refactoring would have been unnecessary.

On the science side, Sabine Hossenfelder clarifies that the proposed "minimal length" in fundamental physics is not discretization or pixilation, but "an unavoidable blur on an image... some kind of fundamental fuzziness that nature has." And Sabine's guest post traces algorithmic undecidability back to Alan Turing's 1936 demonstration that the Halting Problem admits no algorithmic solution; showing a problem is equivalent to the Halting Problem remains the key test of its uncomputability.

Photonic computing, argues anderskaseorg, hits a fundamental wall on NP-hard problems: the amount of light at the output node drops exponentially with problem size, forcing longer detection runs — and thus the running time increases exponentially even in the idealized absence of background noise.

Engineering Culture and Metrics

A collection of engineers weighed in on what matters in practice:

• Debugging tip from @cocoaphony: you must first accept that something you believe about the system is false, because if everything you believed were true, the system would work.
• Wesley Aptekar-Cassels argues that correct software, for any meaningful definition, exceeds current human capability; that peak productivity is closer to 2 hours per day than 8; and that teammate kindness affects effectiveness more than programming language choice.
• Roger Lee's advice to marketplace companies: pay attention to unit economics. He lays out three metrics: break even on paid acquisition within 12 months, maintain LTV-to-CAC of 3X or more within 2-3 years, and respect the "Rule of 40" (growth rate plus EBITDA margin at or above 40%).
• Dan Luu notes that reaching the 95th percentile isn't impressive because it's not that hard — and defends the idea against charges of elitism: most people can become relatively good at most things.
• @kellabyte shares that the only highly scalable system he witnessed doing thousands of requests per second was written in Ruby — with roughly 2,000 instances behind it.

The most colorful anecdote comes from @lethain, recalling Digg's brief salvation: a spike in Facebook activity made them the top-ranked newsfeed application for one day. Their two data scientists ran the analysis. The hero? "A piece of malware pretending to be a Justin Bieber-Selena Gomez sex tape."

Also noted and quotable: Matthew Skelton urges teams to co-design the organization and system architecture. Benedict Evans observes that since software ate the world, "all the world's problems get expressed in software. We connected everyone, including the bad people." Alexander Krylatov says transport engineers lack competencies in system-level traffic performance: local improvements simply rearrange where jams appear. Geoffrey Roberts advises focus: "don't be distracted by new or adjacent markets until you're truly winning and dominating in your own." And @ajaynairthinks credits Kubernetes — whatever its technical merits — with a powerful force in its favor: Hope.

Backend Lore, Old-School Style

A guide to writing backends without containers, Kubernetes, or any of the modern orchestration tooling. The author describes a hybrid cloud approach built on a minimal stack: Ubuntu LTS, Node.js LTS, Redis, NginX, and Amazon's S3, SES, and EC2. The material covers low-level details often glossed over in managed-service era tutorials: environment variables, notifications, code structure, provisioning, database partitioning, deployment, and identity/security. Architecture patterns are graded from A to E, ranging from a simple local development setup to a fully load-balanced, high-availability system. The author's primary guidance on service boundaries: if two pieces of information depend on each other, they belong to the same server.

The stated motivation is to offer a viewpoint that has worked in practice rather than chase under-scrutinized "best practices." The author iteratively pursues simplicity, which has reportedly produced lean, maintainable server code with minimal time investment and hardware resources.

Stateful Functions as the Next Serverless Step

Cloudburst is positioned as a faster platform for general-purpose serverless computing, one that extends the benefits of FaaS to stateful applications. It reportedly outperforms standard FaaS architectures built on AWS Lambda, DynamoDB, and Redis by orders of magnitude. The design principle is logical disaggregation with physical colocation (LDPC), which relies on multi-master data replication to keep "hot" data physically close to function executors for low-latency access.

Implemented on the Anna autoscaling key-value store, Cloudburst combines mutable caches co-located with executors for data locality. Cache consistency is handled via lattice-encapsulated state and new definitions and protocols for distributed session consistency. Benchmarks reportedly show state-management overheads reduced by orders of magnitude compared with current FaaS platforms. The work is available as a paper, an article, and code on GitHub.

HTTP/2 for REST APIs

Performance testing across HTTP/1.1, HTTP/2, and HTTP/2 with server push for REST APIs yields a set of pragmatic conclusions. If raw speed governs the design, compound documents — larger, bundled responses — remain the best approach. If a cleaner, more granular API matters more, many small endpoints are a viable trade-off. Caching has only a minor impact on results. Optimizations in this area tend to benefit the server side more than the client.

Thread-Local Allocation and GC Pauses

A production postmortem details a slow degradation in large systems traced to memory allocation patterns. When allocations form a lattice structure across thread-local storage with many threads, garbage collection pauses can become extremely long. The issue is subtle enough to require significant investigation to surface.

UltraRAM: A New Memory Candidate

Researchers at Lancaster University have developed "UltraRAM," a nonvolatile memory type based on a triple-barrier Resonant Tunneling (RT) structure. It exploits quantum properties to enable low-voltage read and write operations. The technology promises to eliminate flash wear and make DRAM nonvolatile, removing two energy-wasting characteristics: refresh cycles and destructive reads.

Adoption faces a familiar hurdle: the memory relies on chemical elements outside a standard CMOS process. Moving from DRAM's CMOS baseline to III-V materials will likely make manufacturers delay adoption as long as possible.

Discord's Move from Go to Rust

Discord's performance problems with Go stemmed from CPU stalls caused by garbage collection spikes. But moving to a non-GC language introduces its own concern: memory fragmentation in long-lived processes. Sev 1 bugs that appear after months of flawless operation often trace back to allocation failures from fragmentation, sometimes forcing teams to write custom memory allocators anyway.

In load testing, Rust matched Go's latency but eliminated the spikes, even with only basic optimization effort. After profiling and targeted optimization, Rust beat Go on every metric: latency, CPU, and memory. Team-side benefits included strong type safety and a borrow checker that eased refactoring as requirements evolved, plus a quickly maturing ecosystem and tooling.

Related coverage includes Discord's earlier business case for Go, framed around cost savings.

Integrated Systems Over Microservices

DHH's counterargument to microservices is summed up in a phrase: integrated systems. The pitch is that microservices, like TDD, can become a pseudoscientific promise — a paradigm offering absolution rather than a proportionate solution for a given problem. In most cases, the argument goes, the complexity isn't warranted.

The prescription: integrated systems need integrated developers. Engineers who can hold the whole application in their heads and build entire features are preferable to specialization and compartmentalization. The same scrutiny is applied to microfrontends, questioning whether the same fragmentation pattern is repeating at the UI layer.

State of Serverless

DataDog examined usage across thousands of companies. Findings: half of AWS users have adopted Lambda; adoption is higher in large environments; container users have flocked to Lambda; SQS and DynamoDB pair most frequently with it. Node.js and Python dominate the runtime mix. The median function runs for 800 milliseconds, and one fifth of functions complete in 100 ms or less. Half of all Lambda functions run with the minimum memory allocation, two thirds of defined timeouts are under a minute, and only 4 percent of functions have a defined concurrency limit.

Fargate vs. Lambda

A detailed comparison of AWS compute platforms yields scenario-specific guidance. For API workloads, Fargate is the clear winner unless traffic is extremely spiky. API Gateway does more than an Application Load Balancer, providing rate limiting and authorization, though rate limiting is not yet available on HTTP APIs.

For queue-consuming tasks, Lambda becomes far more competitive, even against Fargate with spot pricing. For spiky or low-to-moderate traffic, Lambda's scale-to-zero capability and rapid scaling make it cost-effective, and the SAM framework simplifies application creation considerably compared with Fargate's setup overhead.

Performance testing showed Fargate faster overall, but more important was the consistency of response times. Latency and jitter matter most for time-sensitive or paid APIs; asynchronous workload performance is less critical. Technical limitations keep the platforms from directly competing in some areas: long-running tasks and background processing are poor fits for Lambda, while event sources like DynamoDB streams cannot be consumed by Fargate.

Starlink's Wide-Ranging Implications

Observations on Starlink's potential impact cover a broad spectrum. High-frequency trading firms will likely be the first customers, seeking milliseconds of reduced latency between markets. Some nations will probably ban the system outright to preserve last-mile monopolies and censorship control. Governments where SpaceX staff reside may pressure the company to enable selective censorship or wiretapping.

Multinationals may adopt Starlink as an emergency WAN backup. Automotive manufacturers could embed the "pizza-box"-sized antennas into vehicle roofs, providing onboard Wi-Fi. Airlines and cruise ships will likely use Starlink for backhaul while maintaining current pricing structures. The prospect of practical seasteading in international waters is raised, as is backhaul for new GSM/LTE/5G towers in previously impractical locations.

Shopify's Move to React Native

Shopify's decision to adopt React Native was informed by its acquisition of Tictail in 2018, a mobile-first company fully invested in React Native. Three deep product investments followed in 2019. The company already uses React extensively on the web, making that expertise transferable to mobile development. Shopify sees the framework's performance curve bending upward and plans long-term investment comparable to its commitments to Ruby, Rails, Kubernetes, and Rich Media.

Offline-First Apps and Other Notable Open Source

Replicache is a per-user cache positioned between a backend and its clients. Incoming data placed in the server replica is transmitted as deltas to clients on their next sync, while client-side changes are forwarded as API requests to the service. The tool guarantees that after each sync, the client replica will exactly match the server, making it easier to build fast mobile, web, and desktop applications that are offline-first. As noted by @aboodman, it relies on a mechanism "sort of like a CRDT," though a true CRDT is unnecessary since the system is not intended for masterless environments.

Other projects worth a look include trekhleb/state-of-the-art-shitcode, a tongue-in-cheek primer on bad coding practices; Netflix-Skunkworks/riskquant, a library for quantifying risk; and cdk-patterns/serverless, an example CDK stack based on Jeremy Daly's Scalable Webhook architecture.

Smart LEDs for Privacy and NSA Cloud Guidance

Research presented in "Automating Visual Privacy Protection Using a Smart LED" introduces LiShield, a system intended to deter unauthorized photographing of sensitive indoor spaces. The approach uses smart LEDs modulated with specialized waveforms that are imperceptible to the human eye but interfere with mobile camera image sensors. This enables location-bound visual privacy protection that requires no user intervention and does not disrupt human vision.

On the security side, the NSA has released guidance on Mitigating Cloud Vulnerabilities. The document classifies cloud vulnerabilities into four categories: misconfiguration, poor access control, shared tenancy vulnerabilities, and supply chain vulnerabilities. While cloud customers are central to mitigating the first two, the guidance also covers actions to protect resources against the latter two. Each class is described along with recommended mitigations, framed within a risk-based approach to cloud adoption.

Load-Balancing Inside a Single Machine

Most load-balancing research targets distributing traffic across multiple servers, but RSS++ addresses the problem within a single machine that may have hundreds of CPU cores. The technique dynamically modifies the receive side scaling (RSS) indirection table to place load more optimally across cores. Under high CPU utilization it yields up to 14x lower 95th percentile tail latency and orders of magnitude fewer packet drops than standard RSS. It also supports higher CPU utilization and dynamic core allocation based on input load, removing the typical 25% over-provisioning requirement. Additional materials include an article, video, and associated GitHub repository.

Faster Reads and Outage Diagnosis

FLAIR proposes a method to accelerate read operations in leader-based consensus protocols by leaning on programmable switches. The approach runs a packet-processing pipeline that tracks client requests and system replies, identifies which follower replicas are consistent, and forwards read requests at line speed to replicas that can serve them without giving up linearizability. Results show up to 42% higher throughput and 35-97% lower latency for most workloads compared with state-of-the-art alternatives.

A separate study, "Trade-Offs Under Pressure: Heuristics and Observations Of Teams Resolving Internet Service Outages," examines how engineers actually diagnose outages. The research identifies four heuristics in use: first correlating the faulty behavior with recent software changes; if no correlation is found, widening the search to any conceivable contributing factor; then narrowing possible diagnostic directions by focusing on the one that most readily comes to mind, whether due to a similar past incident or recent events; and finally, favoring peer review over automated testing when making mitigation changes to software.