A different kind of security model

For the past few months we've been running security-focused LLMs against our own repositories to find vulnerabilities before attackers do, and to understand what the latest models are capable of in the hands of someone with malicious intent. No model has drawn more attention than Anthropic's Mythos Preview, which we were invited to test as part of Project Glasswing. We pointed it at more than fifty of our repositories to see what it would find and how it works.

Mythos Preview is a genuine step forward. Previous general-purpose frontier models we've tested could identify interesting bugs and write thoughtful descriptions of why they mattered, but they stopped there—leaving the actual exploit chain unfinished and the question of exploitability open. Mythos Preview operates differently. Two capabilities stood out across our testing:

  • Exploit chain construction — Real attacks rarely rely on a single bug. They chain several small attack primitives into a working exploit, such as turning a use-after-free into an arbitrary read/write primitive, hijacking control flow, and using return-oriented programming (ROP) chains to take over a system. Mythos Preview can reason about how to combine these primitives into a working proof. Its reasoning resembles that of a senior researcher rather than an automated scanner.
  • Proof generation — Finding a bug and proving it's exploitable are different tasks. Mythos Preview does both. It writes code that triggers the suspected bug, compiles it in a scratch environment, and runs it. If the program behaves as expected, that's the proof. If not, the model reads the failure, adjusts its hypothesis, and tries again. A suspected flaw without a working proof is speculation, and Mythos Preview closes that gap on its own.

Some of this isn't unique to Mythos Preview. Other frontier models run through the same harness found many of the same underlying bugs, and sometimes got further than expected on reasoning. Where they fell short was at stitching pieces together—taking low-severity bugs that would traditionally sit invisible in a backlog and chaining them into a single, more severe exploit. That's what changed.

Inconsistent organic guardrails

The Mythos Preview model provided through Project Glasswing didn't include the additional safeguards present in generally available models like Opus 4.7 or GPT-5.5. Still, the model organically pushes back on certain requests, similar to how its cyber capabilities make it useful for vulnerability hunting. But these organic refusals aren't consistent. The same task, framed differently or presented in a different context, can produce completely different outcomes.

In one instance, the model initially refused to do vulnerability research on a project, then agreed to perform the same research on the same code after an unrelated change to the project's environment—nothing about the analyzed code had changed. In another case, the model found and confirmed several serious memory bugs in a codebase, then refused to write a demonstration exploit. The same request framed differently got a different answer; even the same request can produce different outcomes across runs due to the model's probabilistic nature. Semantically equivalent tasks can yield opposite outcomes depending on how and when they're presented.

This matters because organic guardrails, while real, aren't consistent enough to serve as a complete safety boundary. That's why any capable cyber frontier model made generally available must include additional safeguards on top of this baseline behavior, making it appropriate for broader use outside a controlled research context.

The noise problem in triage

Deciding which bugs are real, exploitable, and need fixing now was hard before AI entered the picture. AI vulnerability scanners and AI-generated code have made it harder. Two factors dominate the noise rate:

  • Programming language — C and C++ provide direct memory control and, with it, bug classes like buffer overflows and out-of-bounds reads and writes that memory-safe languages like Rust eliminate at compile time. We consistently saw more false positives from projects in memory-unsafe languages.
  • Model bias — Ask a model to find bugs and it will find them, whether the code has any or not. Findings come back hedged with "possibly," "potentially," and "could in theory," and hedged findings vastly outnumber solid ones. That's a reasonable bias for exploration but a ruinous one for a triage queue, where every speculative finding spends human attention and tokens to dismiss—a cost that compounds across thousands of findings.

Mythos Preview is a clear improvement here. A finding that arrives with a working proof of concept is one you can act on, requiring far less time spent asking whether it's real. Our harnesses are deliberately tuned to over-report so we miss less, which comes with more noise. But at triage time, Mythos Preview's output has noticeably higher quality: fewer hedged findings, clearer reproduction steps, and less work to reach a fix-or-dismiss decision.

Why generic coding agents fail at security work

When we first started AI-assisted vulnerability research, our instinct was to point a generic coding agent at an arbitrary repository and ask it to discover vulnerabilities. This produces findings, but it doesn't produce meaningful coverage. Two reasons explain this:

  • Context — Coding agents are tuned for one focused stream of work: building a feature, fixing a bug, writing a refactor. They ingest a lot of source code, hold a single hypothesis at a time, and iterate against it. Vulnerability research is narrow and parallel, not sequential. A human researcher picks one specific thing—a complex feature, a security boundary transition, or a vulnerability class like command injection—and investigates it thoroughly, then repeats thousands of times across the codebase. A single agent session against a hundred-thousand-line repository covers maybe a tenth of a percent of the surface before the context window fills and compaction kicks in, potentially discarding earlier findings that would have mattered.
  • Throughput — A single-stream agent works on one task at a time, but real codebases need many hypotheses against many components simultaneously. You can push a single agent harder, but eventually you're limited by the shape of the interaction itself. The model used directly in a coding agent is fine for manual investigation when a researcher has a lead and wants a second pair of eyes—but it's the wrong tool for achieving high coverage.

Building a harness around the model

Once we accepted that Mythos Preview wasn't suited to the open-ended "find everything wrong here" job, we started building the harness around it instead. Four lessons emerged from running this work at scale:

  • Narrow scope produces better findings — Telling the model "Find vulnerabilities in this repository" makes it wander. Telling it "Look for command injection in this specific function, with this trust boundary above it, here's the architecture document and prior coverage" makes it act more like a researcher actually would.
  • Adversarial review reduces noise — Adding a second agent between the initial finding and the queue—one with a different prompt, a different model, and no ability to generate its own findings—catches noise the first agent would miss checking its own work. Two agents in deliberate disagreement are more effective than telling one agent to be careful.
  • Splitting the chain across agents produces better reasoning — "Is this code buggy?" and "Can an attacker actually reach this bug from outside the system?" are different questions. The model is better at each when they're asked separately, because each is narrower than the combined version.
  • Parallel narrow tasks beat one exhaustive agent — Coverage improves with many agents working on tightly scoped questions, followed by deduplication, rather than one agent attempting to be exhaustive.

Each observation describes model behavior, and together they describe something that isn't a chat interface anymore. We used Mythos Preview itself to build, tailor, and improve our original harnesses to suit its strengths. Our current harness, which scans code across our runtime, edge data path, protocol stack, control plane, and open-source dependencies, operates in the stages below.

Speed alone is the wrong target

The initial response from security teams to Mythos Preview has been predictable: scan faster, patch faster, compress the response cycle. Some teams are now aiming for a two-hour SLA from CVE publication to production patch. The logic is understandable — if the attacker's timeline shortens, the defender's must too. But faster alone will not work, and many teams are about to learn that the hard way.

Speeding up patch delivery does nothing to change the pipeline that produces the patch. If regression testing takes a full day, a two-hour SLA forces you to skip it — and the bugs introduced by skipped regression testing are often worse than the vulnerabilities being fixed. We saw this firsthand when we let the model generate its own patches: a few fixes addressed the original problem while silently breaking something else the code relied on.

The more important question concerns the architecture that surrounds a vulnerability. The goal should be making exploitation difficult even when a flaw exists, so the window between disclosure and remediation matters less. That means defensive layers in front of the application that prevent the bug from being reached. It means application design where a flaw in one component cannot grant access to others. It means the ability to deploy a fix across every running instance simultaneously, without waiting for individual teams to act.

The same tools cut both ways

We are also aware that these capabilities have a dual use. The techniques that helped us identify bugs in our own code can, in the wrong hands, accelerate attacks against software everywhere. Cloudflare sits in front of millions of applications, and the architectural principles above are exactly what our security products implement on behalf of customers. We will have more to share on what that means in the near future.

Teams conducting similar research can reach us at [email protected].

All Mythos Preview research was conducted in a controlled environment against our own code. Every vulnerability surfaced was triaged, validated, and remediated as needed under Cloudflare's formal vulnerability management process.

Contributors to this work: Albert Pedersen, Craig Strubhart, Dan Jones, Irtefa Fairuz, Martin Schwarzl, and Rohit Chenna Reddy.