A closer look at GitHub’s token redesign

Earlier this year, GitHub rolled out new formats for its authentication tokens. The old tokens were 40-character hex strings that were practically indistinguishable from SHA hashes or other random encoded data, which made it hard to catch compromised credentials with the platform’s secret scanning. The new design tackles that problem with three changes: identifiable prefixes, checksums, and increased entropy.

Recognizable prefixes and a separator

Token prefixes are a standard industry trick—Slack and Stripe both use them—and GitHub is following suit. Each token type now starts with gh plus a letter identifying the kind of token:

The prefix is followed by an underscore (_), which was chosen deliberately: since underscores aren’t in the Base64 alphabet, a token can never be mistaken for a randomly generated string like a SHA hash. The underscore also has a usability perk—double-clicking anywhere on the token selects it as a single unit, unlike with hyphens which many applications treat as word breaks.

Those prefixes alone should cut the false positive rate of secret scanning down to 0.5 percent.

A checksum for offline validation

The prefix helps with identification, but a checksum goes further: it lets secret scanning discard fake tokens without hitting a database. Each token’s last six characters carry a 32-bit checksum, which keeps the random segment long enough to preserve entropy while still giving a solid confidence check. GitHub starts with a CRC32 algorithm, then encodes the result using Base62, padding with leading zeros when necessary.

Entropy stays strong

Generating a huge number of tokens each day—over 10,000 personal access tokens on slow days and up to 18,000 at peak—means entropy matters. For OAuth access tokens, the new format actually bumps entropy from 160 to 178, a modest increase that adds headroom without any change in token length.

Math.log(((“a”..“f”).to_a + (0..9).to_a).length)/Math.log(2) * 40 = 160

Math.log(((“a”..“z”).to_a + (“A”..“Z”).to_a + (0..9).to_a).length)/Math.log(2) * 30 = 178

What token holders should do

If you’ve created personal access tokens or OAuth tokens, the recommendation is to reset them. The new prefixes make secret scanning more reliable, which in turn helps you limit the risk if a credential has already leaked. Personal access tokens can be regenerated from the developer settings; OAuth tokens can be reset via the API.

Service providers that issue their own tokens are encouraged to apply similar design principles and join GitHub’s secret scanning program so their formats can be detected as well.