Ask the Auditor

Anthropic announced a new model called Mythos a couple of weeks ago. They’re not releasing it publicly. The reason: it’s so capable at finding and exploiting software vulnerabilities that they consider it too dangerous to put in the open. Engineers with no security background asked it to find flaws overnight and woke up to working exploits.

That story drove some thinking for me.

I’m building an app. It handles user data. It’s transactional… you know, pay to play. I am not an engineer or security expert. But before I said done, I asked my AI coder to put on a different hat.

“You are a security auditor. Review everything you just built. Tell me what breaks, what leaks, and what someone could exploit.”

What came back stopped me cold.

API keys committed to git in plaintext. All of them. Database tables that any unauthenticated user could write to - including setting an account to admin. Credits minted entirely on the client side, meaning anyone with a little patience could print my app money for free. User emails and full collection data readable by anyone with the public key.

I hadn’t shipped yet. I fixed all of it.

This is the role-play principle at work, and it applies everywhere - not just security. Ask AI to inhabit different lenses and report back honestly on its own work.

You don’t need Mythos. You need the right prompts and the discipline to ask before you ship. Better yet - as you go. I learned that the hard way.

Here’s your MVE (Minimum Viable Evaluation) before anything touches a user: • “You are a security auditor. What’s exposed, exploitable, or leaking?” • “You are a hostile user. How do you break this or get something for free?” • “You are a tech architect. What fails at 10x the load I’m expecting?” • “You are a QA lead. What did I not test that I assumed worked?” • “You are a UX expert. Where does this confuse or lose someone?” • “You are an expert in [domain]. What does a real user in this space actually need?”

Six prompts. An hour. No excuse to ship with holes.