Why feature analytics fail quietly

Tagging a new feature with analytics events looks easy: drop an event at each relevant call site, hand the names to your product analysts, and move on. In practice, that approach tends to produce analytics that are bolted on late, untested, and eventually ignored. When a feature evolves and nobody maintains its event logging, the data slowly becomes unreliable — and decisions made from that data can be worse than having no data at all.

At Dropbox, the mobile team worked with product managers and analysts to define what "good" analytics actually means, then invested in tooling and process changes to make it achievable. The result was a framework built on three principles: analytics must be intentional (answering real business questions), credible (code-tested so they don't break silently), and discoverable (usable by the people who need the insights).

Start with business questions, not event names

Engineers frequently receive analytics requests as a list of events to fire — then treat them as ticketing tasks, not design problems. That leads to event logging chosen because they're easy to record, not because they measure anything useful. Worse, no one asks what the data will actually be used for, so the team can end up deliberately measuring the wrong things.

The fix is to invert the workflow: have stakeholders articulate the questions they want answered before any event is defined. Those questions determine what to log and how.

For Dropbox's photo upload feature on mobile, for example, the team identified questions like:

  • How long does a photo upload take?
  • What fraction of upload attempts complete?
  • How many mobile users upload photos at all?
  • What is the week-over-week retention for photo uploads?

Instrumenting against questions like these beats guessing at event names and also shortens the implementation and review cycle.

Treat analytics as code under test

Unit tests validate that a single function behaves correctly — not that a full user flow produces the required analytics trail. To test analytics across a multi-screen workflow, Dropbox relies on its instrumented UI test infrastructure. Tests drive realistic in-app scenarios while capturing the analytics events those flows touch, then serialize results into JSON for comparison against expected output.

That comparison is a form of textual snapshot testing. The expected behavior gets stored in a text file; when the test runs, a new output is generated and diffed against the file. Any mismatch surfaces quickly and shows exactly what changed. The approach is standard in web frontend testing — React's Jest is a well-known example — but works equally well for validating mobile analytics.

In an Android photo-upload test, the instrumented flow tells the app to tap the upload button, choose a file, and wait for a successful completion. As the test runs, it collects the analytics events of interest via a test rule. One call to verifySnapshot then checks the recorded events against the stored raw-resource snapshot file.

@get:Rule
val snapshotTestRule = SnapshotRule.Creator().create()

@Test
@ExpectedSnapshot(R.raw.test_upload_image)
fun testUploadImage() {
    uploadPhotoButton.click()
    uploadPhoto("test_photo.jpg")
    waitUntilSuccessfulUpload()

    snapshotTestRule.verifySnapshot()
}
@get:Rule
val snapshotTestRule = SnapshotRule.Creator()
    .metricsToVerify(
        "upload_button.clicked"
        "upload.success",
        "upload.start"
    )
    .attributesToVerify(
        "extension"
    )
    .create()
    [
      {
        "event": "upload_button.clicked"
      },
      {
        "event": "upload.start",
        "extension": "jpg"
      },
      {
        "event": "upload.success",
        "extension": "jpg",
        "total_time_ms": "EXCLUDED_FOR_SNAPSHOT_TEST"
      }
    ]

Some event fields aren't idempotent — they differ on every run because the test doesn't mock most back-end systems. Snapshot tests therefore can't assert their exact values. For those fields, the test records the presence of the field itself, marking it with EXCLUDED_FOR_SNAPSHOT_TEST. If the field is renamed or dropped later, the snapshot diff catches the change, preserving the event's shape even when its values can't be pinned down.

Snapshots catch real regressions

Applying the intentional-credible-discoverable process to existing features surfaced several hidden problems in Dropbox's mobile analytics:

  • Duplicate events. Some navigation buttons were logging twice on a single user action. Snapshot tests built around core navigation flows made sure the fix stayed fixed.
  • Unusable values. Sign-in failure events were logging the localized error message shown in the dialog. Dropbox ships in 22 languages, so analysts couldn't roll those strings up into a meaningful breakdown without significant manual work. Switching to a consistent enum of error states — for example GOOGLE_AUTH_FAIL — enabled straightforward visualizations and let engineers see which failures mattered most.
  • Migration safety. As parts of the legacy C++/native stack get rewritten, snapshot tests guard feature analytics from quietly degrading during rearchitecting.

Making the data available is the last piece. Dropbox Mobile keeps dashboards dedicated to each core feature, with graphs answering previously defined questions — like Home screen load time. Together with anomaly monitoring and manual dashboard reviews by analysts and on-call engineers, the aim is that insight gets used over time rather than admired once.

Small discipline, faster releases

Dropbox ships mobile builds on a biweekly cadence. A logging bug introduced today can take two weeks to correct in production. That makes upfront care economical: engage analysts and PMs before coding rather than after, define events from questions rather than guesses, and protect every event with snapshot tests. The price is some upfront process — the payoff is not making product decisions on data you can't trust.