Making Video Intuitive: An Explainer

Making Video Intuitive: An Explainer

On the Stream team at Cloudflare, we spend most of our time tuning video pipelines to make small quality improvements that are nearly imperceptible. Today we're doing the opposite: taking a high-quality video and ruining it as systematically as possible. By breaking things on purpose, we can see exactly which encoding parameters matter most and why.

Our test subject is a 17-second clip of Big Buck Bunny, the classic open-source film available under a Creative Commons Attribution license. We'll grade our results on two simple criteria: smooth motion and perceptible crispness. The video shouldn't stutter, and its important details should remain distinguishable.

Under the hood, video is just a flip book—an optimized sequence of still pictures displayed fast enough to fool your brain into seeing continuous motion. Every compression choice we make is a tradeoff between exploiting that illusion and preserving enough information to maintain it.

The Frame Rate Knob

Frame rate, measured in frames-per-second (fps), is the most direct control over smoothness. At 24fps, each frame stays on screen for roughly 41 milliseconds. Drop to 2fps and each frame lingers for half a second—far too long for the brain to synthesize motion. Frame rate alone isn't a sporting way to ruin a video; humans have a very low tolerance for janky movement.

ffmpeg -v info -y -hide_banner -i source.mp4 -r 2 -c:v h264 -c:a copy 2fps.mp4

Resolution and Bitrate: Buckets and Information

Making fine details indistinguishable offers far more room for controlled destruction. We have choices: codec, profile, bitrate, resolution, color space, key frame frequency—each interacting with the others and each affecting more than just quality. File size and device compatibility also shift with every parameter change. Encoding for a 4K display should look different from encoding for a 2007 iPod Nano.

A useful mental model treats pixels as buckets and bitrate as the information filling them. With too few bits, a pixel becomes an increasingly inaccurate representation of the source. The relationship is complicated in practice, but that's the essence. Our encoder is the budgeting software that decides how to distribute available bits for maximum perceived quality.

The budgets are worth remembering. Uncompressed 1080p video needs about 6.2 MB per frame—roughly 50 seconds of footage per DVD. That's why we compress.

Our source video sits at 7.5 Mbps, divided across 24 frames per second, giving us an average of 312.5 Kb per frame. But frames are not created equal. I frames contain full images and eat far more than the average; P and B frames reference changes relative to other frames and often consume far less. This is why some scenes compress better than others.

Experiment 1: Resolution Only

Let's start by scaling the video down to 140p, the lowest resolution most people ever encounter on video platforms.

ffmpeg -v info -y -hide_banner -i source.mp4 -vf scale=-2:140 -c:v h264 -b:v 6000k -c:a copy scaled-140.mp4

The result isn't shocking yet. We kept the same bitrate budget, but with drastically fewer pixels, the encoder can't spend it all—it fills every bucket and discards the leftover. The video lands at roughly a third of the source's bitrate. On a 4-inch phone screen this looks acceptable. At 40 inches, individual pixel rows become visible, the illusion shatters, and the image turns blocky.

Experiment 2: Bitrate Only

Now let's hold the resolution constant and slash the bitrate budget to 100Kbps—one seventy-fifth of the original.

ffmpeg -v info -y -hide_banner -i source.mp4 -c:v h264 -b:v 100k -c:a copy bitrate-100k.mp4

Fur and grass dissolve into indistinguishable blobs. Interestingly, we didn't hit the 100Kbps target—we overshot slightly. With so many pixel buckets demanding a minimum amount of information, the encoder prefers to fill them a little fuller rather than leave them starving. This is the reverse of the previous experiment, where we undershot because there weren't enough buckets to absorb the budget.

Experiment 3: Extreme Bitrate

Let's drop the budget further to 20Kbps.

ffmpeg -v info -y -hide_banner -i source.mp4 -c:v h264 -b:v 20k -c:a copy bitrate-20k.mp4

This is where the video truly becomes unwatchable. Color mostly vanishes, replaced by grayscale rectangles approximating silhouettes. The encoder begins to make brutal sacrifices somewhere between 102 and 35Kbps, prioritizing structure over color and detail. The moving blocks are a hint about how encoding works internally—more on that shortly.

Experiment 4: Resolution and Bitrate Together

Combining both extreme settings—140p at 20Kbps—should be the worst experience yet. It isn't.

ffmpeg -v info -y -hide_banner -i source.mp4 -vf scale=-2:140 -c:v h264 -b:v 20k -c:a copy scaled-140_bitrate-20k.mp4

This looks almost reasonable, like a tiny version of the 1280 by 720 clip at 100Kbps. We have less information but far fewer places to put it, so the buckets fill nicely. The encoder exactly hits the bitrate target, a solid sign it's satisfied. The output runs just 48 KB for 17 seconds of video.

This won't satisfy anyone watching on a 4K display. But on the 320x240 screen of a 2007 iPod Nano—a 2-inch diagonal—it's nearly indistinguishable from a high-quality source. With 4 GB of storage on some models, this "ruined" video is actually a major improvement in user experience. Raw quality metrics only matter relative to the display they're consumed on.

The Perfect Ratio, Interrupted

There is an optimal bitrate-to-pixel-count ratio for any given resolution that gives the best visual quality at the smallest file size. We accidentally found several candidate ratios during our experiments. But the perfect ratio depends heavily on the source content.

Frames crammed with fine detail demand more bits. Footage that changes dramatically between frames—an action scene versus a static security camera—can't lean as heavily on the cheap P and B frames. Animation with flat colors needs fewer quality-sacrificing tradeoffs than live action. A single video may even combine scenes that force opposite strategies.

What the Encoder Is Doing Under the Hood

The grayscale rectangles we saw moving around were the encoder recognizing repeated patterns within and between frames. Instead of re-encoding the same region, it references previous content and shifts it, playing a shell game with picture segments. These segments, called macroblocks, subdivide each frame into NxN squares. The lower the bitrate, the more obvious this shell game becomes.

FFmpeg can visualize these encoding decisions directly, showing us motion vectors for the macroblocks it shifts. Using our 140p scaled version makes the arrows easier to see.

ffmpeg -v info -y -hide_banner -flags2 +export_mvs -i source.mp4 -vf scale=-2:140,codecview=mv=pf+bf+bb -c:v h264 -b:v 6000k -c:a copy motion-vector.mp4

The codec choice is also the most impactful single knob for bitrate, followed by the bitrate budget itself. These levers interact constantly, and real-world encoding is a feedback loop: the encoder sees how a scene consumes budget and adjusts its strategy accordingly.

Modern approaches such as per-title encoding apply exactly this kind of experiment on a massive scale, testing various parameter combinations to find the best resolution-bitrate pairs for each piece of content.

All the FFmpeg commands we used are above, ready to run against any source video you want to sacrifice. The path to better encoding often runs straight through deliberately worse encoding—seeing the failure modes up close is the surest way to learn which knobs matter.