Reading JavaScript’s intent with graph neural networks

Client-side security is a game of scale. Modern pages load dozens of third-party scripts, each updated on its own schedule, and any one of them can be turned into a vector for supply chain attacks. Manual review doesn’t scale, and simple indicators like obfuscation only tell you that something might be wrong, not what the script is actually trying to do.

Page Shield has extended its detection pipeline with an AI model that classifies the intent behind a script. The model identifies three threat categories — Magecart skimming, crypto mining, and generic malware — and is now available to Page Shield add-on customers via the dashboard’s malicious code analysis view for each monitored script.

Why syntax trees beat text sequences

Training a model on JavaScript is complicated by the sheer stylistic variance of legitimate code. The same function can be written with different quoting, spacing, or control flow, and much benign code is deliberately obfuscated. The model has to balance precision (don’t flag the benign weird stuff), recall (don’t miss an attack), and inference speed.

To handle that variance, JavaScript files are first parsed into concrete syntax trees (CSTs) using the tree-sitter library. Unlike abstract syntax trees (ASTs), CSTs retain comments, whitespace, and even error-repair nodes — necessary for a text editor parser but wasteful for machine learning. Page Shield’s preprocessing filters out those unnecessary elements and normalizes details that don’t affect execution, producing a compact AST-like tree where, for instance, string quoting style is irrelevant and escape sequences are already unescaped.

Naively hashing script content for caching falls apart because any whitespace change breaks the hash. Hashing the parsed tree instead abstracts away inconsequential differences. This matters operationally: Page Shield observes roughly 40,000 scripts per second, but because most are repeats, over 99.9% of reported scripts at any moment have already been seen. Via tree-based caching, the model itself runs fewer than 10 times per minute.

Working from syntax trees also provides tokenization for free. Node leaves’ text becomes the tokens, encoded into a count matrix along with node types. Token texts are lowercased to shrink the vocabulary at the cost of case sensitivity — a trade-off under active review.

Notably, obfuscated code gets no special treatment. The parser simply unescapes characters as it processes input, and the vocabulary includes tokens common in obfuscated patterns, such as double-escaped hexadecimal. The model classifies obfuscated and plain malicious code consistently, and scores on legitimately obfuscated benign scripts remain stable relative to their unobfuscated equivalents.

A message-passing network for code structure

Page Shield’s GNN is a message-passing graph convolutional network (MPGCN). Each node in the tree updates its representation by aggregating information from its neighbors — the parent and child nodes — across multiple layers. A pooling layer then collapses the resulting matrix into a feature vector, discarding explicit edge information so standard fully connected layers can refine the representation. A softmax layer outputs probabilities for four classes: benign, Magecart, cryptomining, and malware.

The implementation uses TensorFlow’s TF-GNN library with Keras as the frontend. One known limitation: TF-GNN lacks sparse matrix/tensor support; the memory and latency cost has the team evaluating PyTorch Geometric as an alternative.

The class probabilities are inverted and scaled to a js_integrity score from 1 to 99, with low numbers indicating malicious and high numbers indicating benign. This matches the output format of other Cloudflare detection products like Bot Management and the WAF Attack Score.

Feeding a model with a minority-class problem

Malicious scripts are rare anomalies. Magecart-labeled samples, for instance, make up only ~6% of the training dataset. The benign class is enormous and internally diverse, with script sizes ranging from bytes to megabytes, mixed coding styles, and partial obfuscation. Malicious payloads additionally tend to be small fragments injected into otherwise valid code.

Data collection strategy splits along these lines. For malicious scripts, the emphasis is quantity: given how few exist, every usable sample is added. For benign scripts, the emphasis is on diversity — variance is the value.

The problem with quantity is annotation time. To filter out near-duplicate benign samples before a human spends minutes on each script, the team generated code embeddings from large language models — StarCoder2 and qwen2.5-coder both worked well — and selected only scripts sufficiently distant from all others by vector cosine similarity (threshold 0.10). For local experimentation, Chroma DB serves as the vector database.

The impact is dramatic. In one bucket of unlabeled scripts, an early evaluation model flagged ~3,000 as malicious — far too many to manually review. Applying the embedding similarity filter reduced the annotation workload to 196 samples, under 7% of the original count. Retraining with just those labels cut false positives by 50%.

Testing beyond the test set

Lab accuracy can be misleading. A recent evaluation model showed macro accuracy and malicious precision near 99%, but that doesn’t capture what happens in production. Three validation layers come into play:

  • Metric uncertainty: The team generates 1,000 bootstrapped resamples from the test set (each a 15% random subsample) to compute standard errors and confidence intervals for precision, recall, and F1. Sub-sampled metrics can shift by as much as 20 percentage points within a 95% confidence range — a signal that the test set itself needs improvement.
  • Offline giant-scale benchmark: The model is run against the full corpus of JavaScript scripts cached by Cloudflare’s network over the last 90 days — nearly 1 TiB and 26 million files. Predictions, latency, and throughput are checked with zero production impact.
  • Shadow mode staging: After offline checks pass, the model runs in staging and then in log-only shadow mode against production traffic alongside the existing model before being marked production-ready.

Compliance angle: PCI DSS v4

A common driver for Page Shield adoption is meeting PCI DSS v4 requirements 6.4.3 and 11.6.1, which take effect March 31, 2025. Both hold companies responsible for approving and monitoring scripts on payment pages where card data could be exfiltrated. AI-driven intent classification makes that monitoring practical, distinguishing between, say, a skimmer and a legitimate analytics library without a human review queue that would stretch into months.