Security: When the Vibes Go Wrong

In July 2025 an agent deleted a company’s production database during a code freeze. The freeze was real. It was also only a sentence in a chat box, which means it was not a control.

Almost every public vibe-coding disaster is one of four patterns: the agent had more power than the task needed, the app works but the back door is open, you installed something you never read, or the agent read attacker-written text and obeyed it. None of these is an argument that AI is bad. In every case a classic engineering control was simply missing — environment separation, access control, least privilege, dependency hygiene, backups. AI removed the friction that used to force people to learn those controls first.

A code freeze that lived only in a chat window

Replit’s CEO, Amjad Masad, called it “Unacceptable and should never be possible.” (The Register) Look at the list he answered with: automatic separation of the development and production databases, staging environments, one-click restore of project state, and a chat-only mode where the agent can talk but not touch code. Every one is a classic engineering control. None is a better prompt.

The bug nobody can see in a demo

On 25 July 2025, users of 4chan found that the Firebase storage bucket behind Tea — a women’s dating-safety app with over 1.6 million users, top of the App Store that week — could be read with no authentication at all. About 72,000 images came out, roughly 13,000 of them verification selfies and government IDs. Three days later a researcher showed that more than 1.1 million private messages were readable through the API. Apple pulled the app in October 2025. (404 Media, Wikipedia)

The generated version of that mistake has a CVE. In March 2025 the researcher Matt Palmer — who worked at Replit at the time, a competitor, which is worth knowing — found a site built with Lovable that let anyone read and write its user table using the public Supabase key that ships to every browser. A scan the next day found 303 vulnerable endpoints across 170 projects — 10.3% of the 1,645 Lovable apps analysed — exposing names, emails, phone numbers, home addresses, payment data and API keys. It became CVE-2025-48757.

Understand why before you ship anything on Supabase or Firebase. The browser talks to the database directly, with a key that is public by design. Row Level Security policies are the only thing between the internet and your tables. An agent asked for “a sign-up form” produces working CRUD and no policies — and the app behaves identically either way. In January 2026 Wiz found the same shape in Moltbook: key in the client bundle, no RLS, unauthenticated read and write, 1.5 million API tokens exposed (Wiz).

Nearly half of unprompted AI code arrives with a known flaw

Veracode ran over 100 models through security-sensitive coding tasks: 45% of the samples introduced an OWASP Top 10 flaw. Java failed 72% of the time, JavaScript 43%, Python 38%; cross-site scripting tasks failed 86%. Security performance stayed flat regardless of model size or training sophistication. Across their three editions the pass rate went 55%, then 55% again (Mar 2026), then 56% (Jul 2026). Syntax correctness has gone to roughly 100%. Security has not moved.

Two honest caveats. Veracode sells security tooling. And their prompts contain no security instructions at all, so 45% is the default, not the ceiling. The ceiling is yours to raise: say “parameterise every query, validate input on the server, escape output” in your prompt or your CLAUDE.md, then review what comes back.

Two kinds of code you install without reading

Models invent package names. Researchers tested 16 code models over 576,000 samples and recorded 205,474 unique names that do not exist; the rate was at least 5.2% for commercial models and 21.7% for open-source ones (USENIX Security 2025). Attackers register the invented names and wait. Seth Larson of the Python Software Foundation named it slopsquatting in April 2025 (simonwillison.net).

The rate improved; the threat did not. A 2026 study of five frontier models over 199,845 prompts found hallucination rates of 4.62% to 6.10% — and 127 fake names invented identically by all five, 53 of them still free to register after disclosure (arXiv). Predictable is all an attacker needs.

The second kind is newer. Agent skills, MCP servers and plugins are text files that steer an agent holding your permissions. In February 2026 Koi Security audited 2,857 skills on one marketplace: 341 were malicious, about 12%, most using a fake “Prerequisites” step to get an information stealer installed (The Hacker News).

You have now seen these patterns as headlines. Here they are as code, which is where you will actually meet them.

The agent reads attacker text and does what it says

A model cannot reliably separate the data it was given from instructions hidden inside that data. Simon Willison’s name for the dangerous combination is the lethal trifecta: an agent with (1) access to your private data, (2) exposure to untrusted content, and (3) a way to send data out. Any two are survivable. All three is an exfiltration tool waiting for the right web page, issue, email or support ticket. On the guardrail products sold to patch this, he is blunt: “95% is very much a failing grade.” (simonwillison.net)

Two demonstrations, neither needing a malicious tool:

• GitHub MCP. An attacker opens an issue on your public repo containing instructions. You ask your agent to look at open issues. It follows the text, reads your private repositories, and publishes what it finds in a public pull request (Invariant Labs).
• Supabase MCP. An attacker files a support ticket containing instructions. You later ask the agent to show recent tickets. Because the connection uses a service_role key, which bypasses RLS, the agent reads the tokens table and writes it back into the ticket thread (General Analysis).

There is also an attack you cannot see. An MCP server’s tool descriptions go to the model, not to you. Invariant Labs demonstrated an innocuous add tool whose description quietly told the agent to read the user’s SSH private key and pass it along as a hidden parameter — plus a server that rewrites its descriptions after you approve it (Invariant Labs). Supabase’s own guidance is short: “Never connect AI agents directly to production data.” (Supabase)

A control is something the agent cannot do

Secrets live on the server. Only public keys — NEXT_PUBLIC_*, the Supabase anon key — may reach a browser. Search your built bundle and git history for sk-, service_role, AKIA and -----BEGIN. Rotate anything ever committed or pasted into an AI chat: between February and April 2026 any logged-in Lovable user could read the chat history of public projects, and chats are full of pasted credentials (Lovable).

Access rules are on and tested. RLS enabled on every table, a policy per operation, Firebase rules deny by default. Then attack yourself — the highest-value hour in this module.

# Log out. Copy the public key from the browser's Network tab.
curl "https://YOUR-PROJECT.supabase.co/rest/v1/signups?select=*" \
  -H "apikey: YOUR_ANON_KEY"

An empty array means RLS is working. Rows mean every stranger on the internet gets the same rows.

Rate limits and spend caps on everything paid. In March 2025 a founder shipped a SaaS built with Cursor, posted about it, and within roughly 48 hours wrote “guys, i’m under attack” — maxed-out API keys and people walking past a subscription check that lived only in front-end JavaScript (techstartups.com). “Nobody would attack my small app” is measurably false: Anthropic’s August 2025 threat report describes one criminal using Claude Code against at least 17 organisations (Anthropic).

Dependencies get checked before they get installed. For every package the agent adds, open the registry page: does it exist, how old is it, how many downloads, does the repository link go anywhere real? Commit the lockfile, run npm audit.

Permissions are enforced, not requested. In Claude Code, put deny rules for .env, ~/.ssh and ~/.aws in .claude/settings.json and audit them with /permissions. Start the agent inside the project folder, never your home directory — in the default mode it can only write inside the folder it started in, and asks before reading outside it. Never use --dangerously-skip-permissions on your real machine: in the August 2025 Nx supply-chain attack, the malware hunted for installed AI CLIs and invoked them with exactly those skip-all flags to find secrets (Wiz).

Know the limit of a deny list. It stops the obvious accident, not a determined path around it — an agent blocked from a command can write a script that does the same thing. That is why the allow list, the working-directory boundary and the sandbox exist as separate layers. Where you want real autonomy, isolate the machine rather than lengthen the list.

Review this repository as an attacker, not as its author.

Find, with file and line for each:
1. Secrets in client-side code, in the built bundle, or in git history
   (sk-, service_role, AKIA, -----BEGIN).
2. Tables or storage buckets reachable without authentication, and any
   missing or permissive row-level security / storage rules.
3. Privileged actions - payment, role change, quota - enforced only in the UI.
4. Unvalidated input reaching SQL, a shell command, or rendered HTML.
5. Dependencies that do not exist on the registry, or that you cannot
   trace to a real repository.

For each finding, write the concrete exploit: the exact request an
attacker sends and what comes back. Do not fix anything yet.
Rank by what an anonymous stranger can do today.

Claude Code also ships a /security-review command you can run on a branch. Treat both as a first pass, never as proof.