The Problem With unittest-Style Assertions

Dropbox's migration from unittest to pytest went smoothly on the surface—pytest is compatible with unittest test cases, so the existing suite kept passing. But the team wasn't getting the full benefit of pytest's assertion rewriting, which produces detailed failure messages by inspecting raw Python assert statements. Tests written in the unittest idiom (self.assertEqual(a, b)) don't go through that rewriting path, so failures remained less informative.

Test output using unittest style assertions:

test/test_login.py:80: in test
self.assertEquals(login.call_count, 1)
E AssertionError: 0 != 1
assert login.call_count == 1

pytest output with raw Python asserts:

test/test_login.py:80: in test
E AssertionError: assert 0 == 1
E + where 0 = <MagicMock name='mock.desktop_login.login' id='140671857679512'>.call_count

The difference matters when debugging failures at scale. With thousands of tests in the codebase, the team had also ended up with mixed testing styles—some files using raw asserts, others using unittest methods—which made the codebase harder to navigate.

Converting Code With unittest2pytest

To close that gap, Dropbox built unittest2pytest, a tool that rewrites unittest assertion calls into plain assert statements. Under the hood it uses the lib2to3 library for automatic code transformation. The tool handled the majority of the conversion without manual intervention, although it ran into trouble with certain whitespace patterns and inline comments—cases it simply skipped rather than risk a bad rewrite.

Hunting Flaky Tests With pytest-flakefinder

Flaky tests are rare failures with real costs: a test that fails 0.1% of the time is tolerable in isolation, but across a suite of thousands of tests it becomes a constant source of noise and lost developer time. Triggering the failure used to require manually looping pytest or pasting test code repeatedly. To make that process reproducible, Dropbox built pytest-flakefinder, a plugin that runs each test multiple times in a row. Paired with -x --pdb, it lets a developer run a suite until the first failure and land directly in a debugger session to inspect the cause.

Both tools are now open source on GitHub. Dropbox uses flakefinder in CI to catch flakiness in updated tests proactively, before it reaches the rest of the team.