Why vaccine data was broken

When VaccinateCA started in January, the covid vaccine rollout was chaotic. Allocation decisions happened at multiple levels of government and private enterprise, and the systems involved did not interoperate. Eligibility rules were a patchwork of policies, frequently overridden by individual healthcare workers. People hunting for shots reported visiting more than a dozen websites and calling 20-plus locations sequentially, swapping tips via messenger.

The project’s founders suspected that no Grand Unified Site was coming from any position of authority. So they started with what they knew worked: build a scrappy hackathon project and see what stuck. The initial call to action was a tweet suggesting this would be a good project for technologists. Within hours a team was assembled on Discord.

The core operational insight was simple: the person actually injecting doses knows whether more are available and who is eligible to receive them. Everything else in the ecosystem — county policy pages, reported stock numbers, allocation databases — risked being stale or wrong. So the group did what any vaccine seeker would do: picked up the phone and started calling pharmacies and hospitals.

From phone calls to a data set

Early calls were logged in Airtable. The first public website was static HTML generated by Jekyll, with the API being a JSON file exported from Airtable on a cronjob. The low-tech approach began paying off within 24 hours, with anecdotal reports of successful vaccinations. Within days it was clear the project was more impactful than expected, so the volunteers formalized into a non-profit, hired some full-time staff, and iterated on strategy.

The result is a data set covering vaccine availability in California that the project says is now the best public one in the state, serving millions of users and helping hundreds of thousands get vaccinated. The operation has since expanded to Vaccinate The States.

The data comes from direct phone intelligence, collation of public sources, and brokering between other projects. Its consumers now include Google Maps — search for covid vaccine locations in California and you may see the project cited as the source — as well as state and county governments.

Working with the official system

As the volunteer effort gained traction, the relationship with official bodies inverted. Instead of being told to stand down, VaccinateCA began receiving requests from California counties and other entities. The formal vaccination effort had its own data problem: governments did not always know precisely where vaccines were administered or under exactly which eligibility criteria. Policy changes often reached pharmacists days late, via newspaper coverage rather than official notice.

The project developed backchannels to report incidents its phone operations found, passing along data sets and analyses as needed. One example: a county had hundreds of doses sitting in freezers because its health department missed a single CSV upload, leaving a dozen pharmacies unable to schedule appointments. No one had noticed because no infrastructure existed to flag it.

The data has been useful to governments that wanted to embed vaccine availability on their own sites. VaccinateCA built an embed for that purpose, which stays current without operational toil beyond the initial integration. When governments asked for it, the project reports a URL was working 30 minutes after the email arrived, was localized into seven languages the following day, and required none of the months-long RFP process typical of government software procurement.

The operation also sits between smaller community efforts and larger consumers. It works with VaccineFinder, the CDC-blessed national initiative run out of Boston Children’s Hospital. VaccineFinder has a pipeline for updates from pharmacy chains’ internal systems; VaccinateCA’s independent status lets it add community walk-in clinics and other sites that VaccineFinder was not designed to accommodate.

The national scaling challenge

The org's national ambitions were slowed by the default strategy of building from zero in each state. Two factors accelerated the expansion: the VaccineFinder partnership, and the assistance of state and county health departments. One unexpected discovery: the most common publishing platform in America for official vaccine availability is a Facebook page. There are thousands of them.

To handle that, VaccinateCA employs what it calls “web bankers” who, with cronjobs and purpose-built UIs, reload those pages frequently, key new vaccination locations into the system, and queue them for direct verification when necessary. Information then fans out from the originating agency’s post to all consumers of the data set: the VaccinateCA site, Google Maps, and participating government entities. Web scraping fills in the gaps for county health departments still running old-school publishing tools.

The effort originally expected to be an open-source project that other states might clone. That turned out not to be viable. The hard part was not software development but the operational cadence: making hundreds or thousands of phone calls a day, sustained for months. Most volunteer projects could not maintain that pace.

A structural problem

The pandemic was pathologically misaligned with how government writes and procures software. The vaccination effort was deliberately administered very locally, leaving thousands of county health departments, most with little software capability, responsible for availability criteria, stock levels, and allocation decisions.

The results were predictable: state and local systems that did not talk to each other, procurement processes giving only one chance to get a system working, and repurposed federal infrastructure built for yearly flu shots rather than daily updates on a rapidly shifting situation.

The project's regrets are forward-looking. “We should have seen this coming,” its leadership says. If a national site had been ready the day the first shot was administered, the country would have been better served. The lesson for the next pandemic: match world-leading scientific speed with the pedestrian web applications that a distribution effort actually needs. It took America’s scientists two days to develop the vaccine; the supporting data infrastructure took far longer to catch up.

Why the Vaccine Rollout Still Needs Engineers

The public narrative around the US vaccination campaign has shifted from disaster to success story — largely because the vaccine is now easy to get for people who write about national politics. That is not nearly good enough.

For Americans underserved by the healthcare economy, getting vaccinated remains difficult. Appointment-only sites fill up within minutes, and scheduled appointments are routinely canceled when promised shipments don’t arrive. The government doesn’t specialize in conversion optimization, and it shows. Primary care providers are often left in the dark about the effort, so patients assume — wrongly — that they’ll be contacted when it’s their turn.

Half the country still has to be vaccinated. The technology industry’s expertise needs to be applied to this conversion problem. It can’t be harder to get a vaccine than it is to buy something on Amazon or to sign up for Facebook.

Barriers to vaccination have to come down everywhere. That means publicizing walk-in sites available outside work hours and surfacing vaccines in underserved neighborhoods that aren’t on official maps — sometimes because of poor organization, and sometimes as a considered policy choice.

The pace must accelerate. Most American lives saved by the vaccination campaign have already been saved, given the disproportionately higher risk among seniors. That shouldn’t cause urgency to flag: many unvaccinated people will still die or suffer severe, sometimes lingering, covid-19 cases. And the longer herd immunity takes, the more disruption to daily life and the economy.

The longer America struggles with covid-19, the longer we postpone helping others. India is in the middle of a national healthcare crisis. Lower-income countries face multi-year timelines for vaccines, and even wealthier nations like Japan and much of the EU are months behind the US. A faster domestic effort means the best parts of the US response can get to work saving lives elsewhere sooner.

Every day matters. Every dose matters.

How to Contribute

The effort is currently expanding the national data set to the coverage of the California data set. The rate-limiting step is ingesting new vaccination locations for contact, scattered across thousands of URLs using hundreds of different CMSes and data architectures.

If you can write scrapers, you can turn those URLs into structured data for the pipeline. The team will run a concordance — “deduping, with style” — against other data sources, queue locations for validation, and publish them to the site and partner outlets. More detail on the process is available on Github.

The team is small and could likely cover 95%+ of sites nationwide within 4-6 weeks without help. A few hours of engineering time can accelerate that. It’s a high-leverage way to help communities emerge from this crisis, and programmers of all skill levels are welcome — this is not rocket science.

The project is also hiring a few mid-career engineers. This isn’t the typical startup gig: no equity, the employer burns through funding as quickly as possible, the TAM declines by millions daily, and success means everyone is unemployed within months. If that sounds appealing, drop a line.

There are also volunteering opportunities for calling pharmacies, joining web bankers, or tackling other tasks.