Card Testing: A Moving Target for Payments Fraud
Card testing—where fraudsters validate stolen card numbers via small or zero-dollar transactions—remains one of the toughest fraud challenges for payment platforms. The attacks fall into two broad categories: verification, where attackers iterate through a known set of stolen credentials, and enumeration, where they guess card numbers within a numeric range. Once a card is confirmed as active, its value spikes: attackers either make larger purchases or sell the details on illegal markets.
The difficulty lies in the signal-to-noise ratio. Card testing is a tiny fraction of overall traffic, and the tactics shift constantly. That makes high-precision detection a demanding machine-learning problem. Stripe’s response is a closed-loop system: rapid detection feeds new labels and features back into models for retraining, which in turn improves detection. Over the past two years, this approach has cut successful attacks by 80%, even as Stripe’s annual payment volume crossed $1 trillion. Illustration by Carolina Moscoso
Multi-Layer Models Set the Block Threshold
At its core, ML-based fraud defense is a probability judgment: above a certain threshold, block the transaction; below it, let it through. The nuance is in determining that threshold dynamically. Stripe applies models at three levels of abstraction:
- Prevalence estimation at the platform level to gauge overall card-testing volume, updating the risk posture daily.
- Attack localization to identify which businesses, issuers, or surfaces are under attack—crucial because a transaction spike might be fraud or a flash sale.
- Transaction-level scoring using a wide array of individual signals.
The outputs are combined to adjust blocking thresholds in real time, targeting a precise slice of suspicious traffic while minimizing friction for legitimate users. Controls ease automatically once an attack subsides. But the system’s accuracy hinges on the freshness of training data, which is why rapid labeling and redeployment are baked into the workflow.
Deriving Labels and Retraining at Speed
Card testing lacks the explicit labels that fraud types like disputes or declines provide. Labels must be inferred. Stripe synthesizes them by consolidating intelligence on newly discovered attack vectors, using automated pattern discovery to combine weak signals, and applying manual expert review. This process turns initially uncertain transactions into confidently labeled fraud cases, enabling proactive blocking of new attack forms within hours of recognition.
Once labels exist, feature engineering must keep pace. Stripe’s Shepherd platform, built in partnership with Airbnb, lets multiple teams generate new features with minimal code. Model validation runs through Flyte, an ML orchestration tool that standardizes experimentation. New candidates are retrained, evaluated on offline data, and compared against the incumbent model via blue-green tests before deployment—all within an incident-response timeline.
Foundation Models for Subtle Patterns
The flywheel approach excels at recognizing historical patterns but struggles with fully novel or highly subtle attacks—especially on large businesses where fraud blends into heavy transaction volume. To close that gap, Stripe augments its specialized models with a large transformer trained on billions of global transactions. This model surfaces patterns that simpler systems miss and encodes payments into atomic embeddings. These embeddings support downstream uses such as training classifiers on embedding sequences to detect in-progress attacks in real time.
The net result is a defense that adapts continuously: successful card-testing attempts are down sharply, while false-positive rates remain low. The system’s architecture—spanning rapid labeling, streamlined feature iteration, and a foundation-model layer for generalization—keeps the platform ahead of attackers who constantly revise their methods.
