Learning Hard Things: A DNS Case Study

Julia Evans recently gave a keynote at RubyConf Mini titled “Learning DNS in 10 years.” The talk isn’t really about the Domain Name System itself. As Evans points out, it’s a Ruby conference and the real subject is how to approach learning something difficult. DNS just happens to be the example that took her roughly 16 years from first registering a domain to feeling like she genuinely understood the system.

That timeline is normal, she argues, because her approach is to do nothing for months at a time and then commit to a furious, short learning session. Thirty minutes, three hours, an afternoon. Then back to doing nothing. Over the years, these tiny deep dives accumulate into real understanding.

Why DNS Is Worth Learning

For those unfamiliar, DNS translates domain names into IP addresses. When you open a site like www.example.com, your browser uses DNS to find the server’s address. Evans sells the topic by noting it’s an invisible system that controls the entire internet. If it vanished, everything would collapse. That kind of behind-the-scenes infrastructure is fun to understand.

DNS is also remarkably old. The core specification, RFC 1035, was written in 1987. The basics haven’t changed since before she was born. So if you’re slow to learn it, she says, take your time — it’s not going to change out from under you.

Spy on the System

Evans’ first technique for building understanding is to watch the system at work. For DNS that means two tools: dig and Wireshark.

dig makes DNS queries from the command line. Running dig maps.google.com prints out a few key fields:

  • The domain name
  • The Time To Live (TTL) — how long to cache the record
  • The record type, where A stands for address
  • The content, which is the IP address

There’s also the amusing class field. Its value is IN, for “INternet.” In 1987 designers anticipated many networks, so they made it a configurable choice. In practice every DNS query now uses the internet class. Other classes like CHAOS and HESIOD exist but are effectively unused.

One fun part of dig is poking around publicly visible records. TXT records, for instance, are commonly used for domain verification. Companies set records like google-site-verification to prove ownership. Because those records are public, you can see which third-party services a company uses. Evans notes that looking at Twitter’s TXT records reveals they work with Miro, Canva and Mixpanel.

By default dig’s output is noisy. She recommends a configuration file with +noall +answer so responses show only the interesting bits.

Wireshark is her favorite tool for inspecting packets. When you make a DNS query, Wireshark shows you exactly what’s on the wire. Looking at the query section in the packet, there are really only two important fields: the domain name and the record type. A DNS query is simply asking, “What’s the IP address for example.com?” That simplicity, she says, makes the whole thing feel less intimidating.

A caveat: encrypted DNS will keep Wireshark from seeing your browser’s queries. Plenty of unencrypted DNS is still available to inspect.

Notice When You’re Confused

Evans told a story she calls “the case of the mysterious caching.” Her mental model of DNS was simple: if she set a TTL of five minutes when configuring a record, she’d never wait more than five minutes for changes to propagate. Resolvers are caches, after all, and they get their data from authoritative servers.

One day she set up a new subdomain, refreshed the page, and it didn’t resolve. Five minutes passed. It still didn’t work. Her model was broken.

Often people let that go — it’s not worth a deep investigation. But this time she had the energy to find out why. After some furious Googling, she found a Stack Overflow comment mentioning “negative caching.”

The issue was that she had visited the domain before setting up its records. The DNS servers had returned an NXDOMAIN response — a 404 for DNS — indicating the domain didn’t exist yet. The resolver had then cached that negative answer. She wasn’t waiting for a positive record to appear; she was waiting for the cached nonexistence to expire.

Read the Specification

Her next question was exactly how long she had to wait. When you have a specific question, she suggests bringing it to the specification — the RFC (Request for Comments). Some of the main DNS RFCs date from 1987, so the comment period is long over. But they remain the ultimate answers to many questions. Since Stack Overflow comments aren’t always trustworthy, she went to the authoritative source.

RFC 2308 contains the relevant sentence: the TTL of a negative answer is set to the minimum of the SOA record’s minimum field and the SOA’s own TTL. It indicates how long a resolver may cache a negative answer.

When she looked at a real example by running dig +all asdfasdfasdfasdfasdf.jvns.ca, there was the NXDOMAIN response along with the SOA record containing two relevant numbers. Both were from an authoritative record with a 10800 TTL. Her wait was 10,800 seconds — three hours.

She waited the three hours, and everything worked. The value of the exercise was knowing exactly how long to wait, rather than following generic advice to allow 48 hours.

The practical solution to avoid this problem entirely is simpler: don’t visit a domain before its DNS records are set up. Only visit it after configuration is done. Evans says this tip almost eliminated the problem for her.

Run Live Experiments

Another strategy is to experiment with the system rather than just reading about it. Most people don’t want to risk breaking their own domain, however. Evans and her friend Marie built Mess with DNS, a site where anyone gets a disposable subdomain (like chair131.messwithdns.com) and can set records freely. If something breaks, it’s someone else’s problem.

Experimenting there revealed something interesting about how caching actually behaves. She made about 20 queries for a record against Cloudflare’s resolver (1.1.1.1) and expected the server logs to show a single request — the others would be served from cache. Instead, the server received eight queries. Repeat experiments with Google’s resolver showed around four queries.

The explanation is that major resolvers are distributed networks. The particular machine her queries hit wasn’t always the same one, so a resolver effectively has multiple independent caches. That complicates the naive model of DNS caching and shows why experiments matter. Her live demos sometimes give results that differ from what she’d see testing at home.

Write Your Own Terrible Implementation

Evans’ favorite technique is implementing a deliberately poor version of the thing she’s learning. For DNS from scratch in Ruby, it’s easier than it sounds.

The first step is trivial: four lines of Ruby connect to Google’s resolver on UDP port 53. Then you need a DNS query. Rather than write the binary format from memory, copy one that already exists — Wireshark can export a query as a hex stream. Paste that hex into a Ruby program, convert it with .pack, send it, and the server accepts it without complaint. The server doesn’t know you have no idea what the bytes mean.

But understanding the bytes is the point. A DNS message has two parts: a 12-byte header and a question section. The header encodes six numbers:

  • A query ID (two bytes)
  • A flags field
  • A question count
  • Answer count and two other empty section counts

In Ruby, an array holding those numbers — with the question count set to 1 and the others to 0 — and using .pack with the n directive to encode big-endian two-byte integers produces the header.

The question section encodes the domain name with length prefixes: example.com becomes 7 example 3 com 0. A couple of final fields specify type 1 (an IP address) and class 1 (“internet”). Encoding this is a short Ruby snippet that splits on dots, prepends each segment’s length, and appends the terminating zero.

The resulting 120-line script successfully queries real domains and parses the responses. Evans has published the full walkthrough in a blog post, “Making a DNS query in Ruby from scratch.” Writing your own terrible implementation confers, in her words, an unreasonable amount of confidence. When a broken-by-design implementation works at all, seeing it work with your own eyes is unmatched, and there are no production consequences to worry about.

Five Techniques for Learning Difficult Things

The talk closes with a recap of the strategies for learning something hard:

  1. Spy on it. Look at what’s really in the bytes. It’s often less complicated than you fear, and that builds confidence.
  2. Notice when you’re confused. You won’t always have the energy to investigate, but when you do, it’s often one missing fact rather than missing everything.
  3. Read the specification. Few feelings beat knowing you have the right answer because you checked the RFC.
  4. Experiment. Testing your understanding is fast, cheap, and frequently proves your model was wrong.
  5. Implement your own bad version. It’s the most work, but the confidence payoff is enormous — and there are no consequences because it’s never running in production.