Cookie banners are where many people express their privacy preferences online, and they’re a visible part of any company’s privacy program. For organizations running a large, evolving set of websites, keeping those banners working correctly is harder than it looks. A banner might behave properly at launch, but a later change to a page’s settings, a new integration, or an experiment can quietly alter how a visitor’s choice is saved or enforced.

Manually verifying that every banner still honors a user’s decision across all of those pages would burn hours of quality-assurance time. Dropbox’s engineering team built an in-house cookie auditor to automate the checks. The tool visits web pages like a privacy-conscious user and confirms that only cookies consistent with that user’s expressed preferences are loaded. That turns what would be an endless manual chore into a scalable, repeatable process.

Cookies fall into two broad categories. Strictly necessary cookies handle essential functions such as keeping a user logged in or remembering language settings, and they load automatically. Non-essential cookies support analytics, marketing, performance, or advertising, and in some jurisdictions require consent before they can load.

Dropbox operates more than 200 web surfaces across products and teams, each with its own cookie profile. URLs are in constant flux as pages launch, retire, get redirected, are localized, or enter experiments. Cookie banners aren’t the only way users communicate preferences either. Global Privacy Control (GPC) lets visitors set a browser-level signal that websites must detect and respect.

Before writing any code, the Privacy and Engineering teams had to agree on what “correct” looked like. Legal terms like opt-in, opt-out, and strictly necessary need human interpretation, but an automated auditor requires concrete outcomes: what should happen when a user makes a given choice, and what should be flagged when behavior deviates. The teams also had to decide which pages to audit, when a missing banner counted as a violation, and which third-party cookies or artifacts should be tracked separately.

Those classifications deliberately live outside the auditor’s source code. Keeping them in configurable lists lets the Privacy team approve or remove cookies and exceptions without waiting for an engineering release. It also lets the system track evolving services and regulatory guidance more easily.

Because Dropbox built its own consent infrastructure rather than using a third-party banner product, the auditor could hook directly into that existing system. That integration gave the team control over how the two worked together and made it simpler to adjust what was tested as requirements shifted.

Auditing the real user experience

The auditor behaves like an actual visitor. It uses Playwright, a browser automation library, to open each page in a fresh, isolated browser session and watch what happens from the moment the page loads.

Each audited page runs through three separate tests, simulating a standard US visitor, a visitor in the EU, and one sending a GPC signal. Every test starts with a clean slate, no cookies and no saved preferences, so the auditor sees the same first impression as a new user. Before touching anything, it records which cookies have loaded and checks them against expectations for that test scenario.

The next step is finding the page’s consent controls and declining non-essential cookies. That is more involved than clicking a button labeled “Decline.” Consent controls may surface in a banner, a floating widget, a preferences window, or a footer link, and they can appear in any of 22 supported languages. Rather than parsing button text, the auditor locates and interacts with the underlying consent elements.

After declining, the auditor reloads the page to verify the preference stuck. It checks that the cookies present still match what should load for that test and records anything unexpected. Judging the system by what actually happens on the page, instead of what a configuration file says should happen, was one of the project’s defining design choices.

Finding the pages that matter

Once the auditor worked, the team hit a second problem: it could only test URLs that were already on a known list. New marketing pages, blog posts, and help center articles go live weekly, and no one on the Privacy team had full visibility into every new web surface. Asking teams to manually submit each new URL would recreate the bottleneck the auditor was meant to eliminate.

The solution was a companion URL detector, effectively an auditor for the auditor. The cookie auditor verifies that a decision is honored; the URL detector verifies that every place needing a consent experience is actually being checked.

The detector started with traffic logs. Every visit to a Dropbox property generates a record, so the initial dataset ran into the billions of rows. Most of that was repetition, the same URL path appearing thousands or millions of times. After filtering out duplicates, the team had a far smaller list of unique paths that was practical to work with.

That smaller list went through a second round of filtering. Pages that didn’t need testing were excluded, pages sharing the same consent logic were grouped together, and representative URLs were chosen from clusters of near-identical pages. The result was a manageable audit set that still covers the range of consent experiences a user might encounter.

The split also reflects a broader principle Dropbox applies to automation: use tooling to surface the relevant evidence so humans can make the final call. Privacy and Engineering teams still review unusual pages and exceptions, but they no longer spend their time searching for them.

Privacy needs continuous validation, not one-off reviews

Privacy programs are built on policies, controls, and review processes, but users ultimately judge them by how products behave. When someone makes a choice about their data, they expect the product to honor it: the choice should be clear, declining should be as easy as accepting, the preference should be remembered, and it should apply consistently as the user moves across a website.

This is why we treat privacy like reliability or security. It demands ongoing attention as products evolve. Building the auditor made it clear how many components must come together to make that possible: identifying which pages to test, separating genuine issues from false positives, translating legal requirements into behavior the auditor could verify, and testing the user experience rather than trusting system configuration.

That perspective shifted how we thought about the auditor itself. It became a way to turn privacy commitments into something we could regularly check against the actual experience on our websites. For us, maintaining that trust in a changing product depends on making clear commitments to users and building systems that continuously verify those commitments are being met.

Weekly reports and a path toward faster triage

Today the auditor provides our Privacy and Engineering teams with a weekly report on consent behavior across Dropbox websites. It flags likely violations while filtering out known false positives, so teams can focus on what needs attention. Historical results also help track trends and reveal when a previously correct behavior starts to drift. Our next step is streamlining the process further by linking findings to the teams responsible for those pages and routing potential issues to the right people faster.

For teams thinking about a similar effort, browser automation may be the easiest part. The real work happens around it: defining what correct behavior looks like, maintaining a dependable inventory of what needs testing, distinguishing real problems from noise, and building a review workflow that brings the right people together. Investing in those areas has given us confidence that our cookie program works as intended, even as the websites and surrounding systems keep changing.

Acknowledgments: Special thanks to folks on the Privacy Engineering and Legal teams for guidance throughout the project.