The VibeSec Reckoning

Why “just prompt it to be secure” fails

Vibe coding has put application development into the hands of citizen builders who could never have written software before. But when our AI applications team in Global Marketing at Thoughtworks was asked to take a vibe-coded video assembly prototype—built with Gemini, Replit AI, and Claude AI—to production scale for 10,000 employees, we hit two near-misses that stopped the work cold. In both cases, the AI suggested a path with serious security consequences, and only a human asking the right question caught it before code ran.

The first incident involved storage access. The AI recommended making the storage bucket public or setting cloud file storage to “anyone with the link.” When challenged, it justified the choice by claiming every company does it. Only a firm rejection prompted a secure alternative. This could have leaked sensitive unreleased brand assets and audience data to the public internet.

The second involved over-permissioned credentials. A service account was assigned the Access Token Creator role, granting it the ability to create short-lived tokens and reach databases and other resources far beyond the task's requirements. The team caught it before running anything, but a compromised service account with that scope could have moved laterally through an entire cloud workspace.

The scale of the exposure

These are not isolated incidents. Attacks exploiting application vulnerabilities are up 44% year on year. One in five enterprise breaches is now caused by AI-generated code. A quarter of AI-generated code carries confirmed vulnerabilities. And half of organisations have no sensitive data policies covering AI use. AI-assisted coding at speed is creating systemic security exposure across industries. The path of least resistance that AI agents naturally take is not usually the secure one; we need controls beyond human judgment.

Prompts are suggestions, not gates

When we shared these incidents with engineering and security colleagues, the message was unambiguous: telling an agent to be safe is not enforcing that it is safe. Prompts are overridden, misunderstood, or ignored when a user pushes back or phrases a request differently. One engineering leader put it bluntly: if you absolutely do not want something to be true, it must be codified in non-negotiable rules somewhere in your development lifecycle. Prompting for test-driven development is not the same as enforcing code coverage thresholds in your build tool. One is a suggestion; the other is a gate.

Birgitta Böckeler’s work on harness engineering makes this concrete. Instead of relying on prompts alone, wrap the agent in an outer “harness” structured along two axes. Guides are feedforward controls that steer the model before it acts; Sensors are feedback controls that observe code after the agent acts. Computational controls are deterministic, fast, and CPU-run—linters, test suites, deployment gates. Inferential controls rely on semantic analysis and AI judgment, such as system prompt constraints.

Business functions carry the same obligations

Marketing and other business functions building with AI are not exempt from the security obligations that apply to engineers. Building security into software is fundamental to protecting customer and employee data, and even lightweight internal prototypes must comply with enterprise standards. Compliance frameworks like ISO 27001 are often contractual requirements; all applications, however quickly built, must meet those benchmarks to maintain client trust. Brand assets—unreleased campaign materials, financial data, audience insights—need protection that over-permissioned service accounts put at risk. When business functions lead with security discipline, they signal responsible AI adoption to the wider organisation and to clients.

Immediate habits to adopt

You do not need a security background to start building responsibly. Three habits will get you moving in the right direction.

Feed security rules into every session

Add your organisation's security guidelines as “Rules” in Claude, Cursor, or Replit to begin with; later, invest in a shared default layer across all tools. The agent uses them as guidance, making secure patterns more likely from the start. These must still be backed by deterministic checks that fail unsafe code, exposed secrets, broad permissions, vulnerable dependencies, or insecure infrastructure before anything ships.

Question AI-Suggested permissions

If a tool recommends making something public or assigning a broad service account role, stop and ask why. The path of least resistance and the secure path are rarely the same.

Run a red team prompt

Ask the AI to roleplay as a bad actor and pen test what it just built. This surfaces vulnerabilities that forward-looking prompts miss, particularly around permissions and data exposure.

Two structural fixes we put in place

These incidents led us to two practical initiatives—both replicable by any team building with AI, regardless of technical background.

A versioned security context file

We compiled technical security rules into a structured context file loaded into every AI coding session before any code is written. Coverage spans zero trust enforcement, secrets management, harness engineering, and supply chain integrity. The file is versioned, loaded by default, reviewed, and paired with automated checks—that operational discipline is what separates it from a casual prompt. It acts as an inferential guide telling the agent what good looks like, but it must be paired with computational sensors in the pipeline to validate output.

The file contains non-negotiable rules that require the agent to decline requests violating policy. If asked to bypass a check, disable logging, or set something to public, the rules should steer it to refuse and explain why. But the critical control is that deterministic checks and deployment gates still catch the issue if the agent fails to follow that guidance. That refusal was exactly what was missing in both of our near-miss incidents.

A daily security intelligence feed

We built an automated consolidation that ensures supply chain alerts reach us the day they are published. In the future, as we work toward an agentic enterprise, we envision agents proactively creating story cards and patching known vulnerabilities behind the scenes for human review, cutting Software Development Lifecycle (SDLC) cycle time significantly.

Staying ahead of the disclosure curve

Continuous monitoring is itself a control. The workflow described in this guidance watches the toolchain and languages your team actively uses, aggregating new CVEs, platform advisories, and vendor security bulletins into a daily digest. Coverage concentrates where risk clusters: your coding languages, your cloud deployment targets, the AI coding assistants themselves, plus the full CVE database.

The objective is to learn about a vulnerability on the day it becomes public rather than weeks after. With 42% of new enterprise software now AI-generated or AI-assisted, the same tools that speed up development are increasingly the subjects of fresh CVE disclosures. Active monitoring is therefore part of owning that attack surface. Both this and the governance framework described above require no engineering background to adopt — one is a machine-readable policy document plus an automated search. What unifies them is the premise that passive awareness no longer suffices when code is being produced at algorithmic speed.

Making the guardrails structural

Lasting change means shifting security from a set of human reminders to embedded control logic. The recommended moves all aim at that same end.

From prompts to feedback loops

Harness engineering should be folded into standard prototyping templates. That means abandoning open-ended probabilistic prompts in favor of deterministic correction loops: when an automated scanner trips, the agentic flow must oblige the model to iterate on its own output until the tool passes.

Rules as code context

Collate your organization's technical security rules into a formatted markdown file and register it as a Rules document in the application builder (Cursor, Claude, and similar). Doing so positions the guardrails where they can catch the most common mistakes before anything is committed to version control, when remediation costs are lowest.

Secure defaults, not footnotes

Delete the friction between doing things correctly and doing things quickly. Enable builders with starter templates that already carry authentication patterns, private storage defaults, secrets handling wrappers, and dependency scanners. If the secure route is also the default route, deadline stress stops being a reason to bypass controls.

One harness, all functions

A shared starter harness built jointly between business units, engineering, and security gives every developer a hardened foundation from the first sprint. Alternately, each squad independently rediscovers and re-introduces the same avoidable issues.

The concrete case: from hackathon to rollout

The thinking here was forged during an engagement supporting another group's video assembly platform for a Global Marketing hackathon. Scaling that prototype made visible what "vibe coding" without enterprise guardrails pulls into an organization.

After the team moved its technical security rules inside the agent workflow itself, those early near-misses had no path into production. What launched was a secure, production-ready platform, used by 150 people for the duration of the event.

The operative shift was the target of control. Working backward from the assumption that humans alone would spot flaws, the team replaced it with an architecture where technical rules, automated checks, and human responsibility are fused into the same pipeline. That template now serves as the blueprint for speed with rigor in an era of agentic output.

Related work and further reading

The problems are not exclusive to this project; Thoughtworks colleagues have published overlapping findings that directly amplify the points above.

• CTO Rachel Laycock on engineering rigor in the AI era: it relocates rather than vanishes. Comprises agent leash patterns, zero-trust designs, and the case for mandatory guardrails.
• A Thoughtworks-hosted enterprise practitioners' gathering observed security losing priority in AI rollouts. A named risk: granting agents unchecked tool access, especially email.
• Birgitta Boeckeler's definitive reference on harness engineering: how to enforce deterministic rules inside the agent loop without freezing model utility.
• Guidance on embedding security rules as pre-commit hooks for any AI-assisted development flow.
• The Thoughtworks annual technology outlook warns that without oversight, generative tools compound debt and expose vulnerabilities. The prescription is co-construction: humans ensure AI-produced code preserves architectural integrity.

Acknowledgments

The authors thank the team for shaping discussions. Credit goes to Julie Woods-Moss for championing AI prototyping risks, and Natalie Drucker for writing contributions and title. Martin Fowler advocated shifting mindsets from short-term to long-term habit building. Kief Morris anchored the argument about LLMs bypassing prompts. Jim Gumbley supplied governance and ISO27001 focus, Craig Stanley argued for staged rollout, and Andherson Maeda proposed the shared skills repository. Simone Thompson contributed the adversarial-simulation method, Premanand Chandrasekaran explored limits of LLM policy enforcement, and Matteo Vaccari balanced strictness against model flexibility. Bilal Jaffery also supported this work.

GenAI assistants (Gemini and Claude) helped accelerate research and polish the writing for clarity.