Continuing the same article:

What an LLM pentest orchestrator actually does

Open-source frameworks orchestrate AI agents through the stages of the traditional Cyber Kill Chain at scale. ARTEX and tools like it can plug into existing Model Context Protocol (MCP) servers and wire standalone pentesting tools together in new combinations, with some allowing operator direction over the type of task to pursue. They map and enumerate environments from both inside and outside, projecting assets onto a graph so that multiple independent parallel agents can enumerate, verify, prototype and attack at the speed of inference. The result is that a compromise becomes a hill-climbing problem — the class of problem agents handle well.

Cloudflare is tracking more than 50 AI LLM agents. The top five LLM penetration and exploitation orchestrators are each fully autonomous and each has over 1k GitHub stars, which amounts to a doctrinal shift in offensive capability:

  • Human skill is no longer the constraint: a harness executes every step of the OODA loop — observing the environment, orienting toward valuable or vulnerable targets, deciding on an action, and acting autonomously.
  • Speed: multiple parallel agents pursuing a single goal differ sharply from a lone operator or a team augmented by AI.
  • Scale: one operator can target several organizations simultaneously, widening their blast radius.
  • Exposed mistakes get found: autonomous frameworks relentlessly and systematically test exposed services, configuration included.

Reading the network: a hydra effect

As of November 12, ARTEX and its forks account for more than 60% of known AI penetration testing orchestrators. Since the original repository was removed on October 8, traffic has grown more than tenfold, and it is up more than 30x from October 1. The takedown and its widespread coverage most likely produced a hydra effect: removing the original repository triggered a surge of forks and modifications, visible as a steep acceleration in unique instances around October 8.

Credential chaining in observed incidents

Over the past eight weeks, Cloudforce One has been called into multiple incident response investigations involving ARTEX and frameworks derived from it, and has seen exposed credentials chained with LLM-based exploitation to outpace defenders. In several incidents the actor gained initial access to a hyperscaler environment and pivoted straight into the customer's Cloudflare environment. Root-cause analysis pointed to a common credential hygiene failure: static, non-expiring tokens running in CI/CD pipelines that carried broad, cross-environment access across the organization's hyperscaler infrastructure. LLM frameworks greatly expand the likelihood that a client's API key will be leveraged laterally to take over other tenant environments.

A worked example

In one investigation, the team observed ARTEX reaching a compromised account-level API token previously used for CI/CD, with no IP restrictions and no rate limiting; the available evidence indicated the compromise most likely originated through a third-party hyperscaler. A large number of new Worker scripts appeared under distinct names such as artex-*, artex-pgrecon-*, artex-pgw-* and artex-do-*, each name correlating to a distinct behavior and often postfixed with an alphanumeric identifier. Activity exceeded 69 requests per minute across several distinct parts of the customer's Cloudflare infrastructure, along with 400 database dump requests within an hour. The team also observed attempted persistence mechanisms and attempts to validate session tokens for other third-party applications. Much of this activity appears to have been only lightly guided by a human operator — a single illustration of the speed and breadth these frameworks reach after a compromise.

The Modern Security Model

Because an LLM can validate each stage of the Cyber Kill Chain and feed the result into the next, a threat actor gains speed and confirmation at every step. That orchestration outpaces legacy cyber security models, so a containment-first architecture is needed instead. The defense below is organized around the metaphor of an estate: one layer of control per step in the attack chain.

Perimeter: traffic control and advanced WAF rules

The first job is to keep the attacker off the property at all, because Layer 3/4 volumetric floods and direct IP probing arrive before any application request does. Network traffic control routes inbound traffic through distributed outer gates, filters out rogue traffic, and severs rogue tunnels so that only legitimate requests reach the origin address.

Network traffic control serves an important role in stopping volumetric attacks.

The foyer: anomaly detection at the edge

Requests that reach the application boundary are often not simple brute force. Automated probes disguise their paths — for example //front-door//back-room/ — to trick routing logic into opening something it shouldn't. The Web Application Firewall enforces clear rules at this point:

  • A client cannot send 500 requests in five seconds (rate limits).
  • A client cannot obfuscate its path to bypass controls.
  • A client cannot attempt to extract diagnostic heap dumps from the application.
  • Credentials should be backed by security keys, including hardware-based FIDO keys.

Rate limiting and anomaly detection are what stop application exploitation here.

Identifying synthetic visitors: bot defense

Automated agents, headless browser scrapers and AI reconnaissance tools imitate legitimate visitors, but their behavior gives them away: mechanical cadence, synthetic micro-behaviors and poorly imitated client identification. Bot Defense and challenges inspect that behavior and remove the machines from the property before they interact with the application.

Removing the back door: identity-gated access and micro-segmentation

Internal admin panels, /actuator diagnostic endpoints and remote gateways are the side doors and cellar hatches of an application estate; locking them is not enough if they remain reachable from the Internet. Zero Trust application access closes the doorway from the outside and limits which internal resources can be entered. The endpoint is effectively invisible: there is no lock to pick and no way to reach it directly. Entry requires an identity check and a device health check, after which the user is placed directly into the specific application they need, with the rest out of view.

security-model.png

For users who are admitted, micro-segmentation prevents lateral movement from the room they entered into the rooms beside it. It also limits what compromised credentials can reach, keeping an intrusion out of production, test and development systems alike.

Inspecting inbound payloads: schema validation and explicit controls

API traffic is the estate's daily delivery, and it must be inspected against a manifest rather than trusted on arrival. Unvalidated payloads and forged requests aimed at restricted internal systems are the common bypass attempt. The controls are:

  • Every inbound request is validated against an OpenAPI schema so it conforms to expected definitions and structures.
  • Clients present mutual TLS (mTLS) certificates to establish trusted, authenticated connections.
  • Requests that fail inspection or lack valid credentials are rejected at the boundary.
  • Every positive security implementation ends with an explicit default-deny rule at the bottom of the ruleset.

Visibility through threat intelligence

Hardened perimeter defenses still leave an organization blind to a technique first deployed elsewhere. Threat intelligence provides that warning: when a new command-and-control infrastructure appears or new tunneling activity starts in the region, the organization receives advance notice, updates its watchlists and sensors, and blocks the traffic before it reaches its own address.

Containment as the response to orchestrated attacks

LLM-based frameworks such as ARTEX put pressure on legacy defense models, and the historical defense-in-depth approach built around a network DMZ no longer covers modern architectures. Securing legacy equipment and on-premises devices calls for an identity-gated approach that makes those devices logically unreachable from the Internet. Beyond identity, the goal of micro-segmentation is to slow lateral movement and make it noisy — a containment-first posture suited to an environment where users consistently lose their credentials. Combined, these elements form the Modern Security Model.

For an active incident requiring on-the-ground response or threat intelligence support, Cloudforce One can be contacted immediately. Help implementing the model described here is available from the Solutions Engineering team.