I gave the same talk a few times this year on "The three elephants in the room for Web3 teams": weak operational security, vibe coding, and AI audits. The first two get talked about openly. The third one (AI Audits) gets sold to you as the responsible choice.
Founders may think: A human-led audit is costly and can sometimes take weeks to even start. I'll say founder as shorthand, but the person who actually makes this call is whoever signs off on the security of the product/service. At a small team that's a founder or a technical cofounder. At a bigger one it's a head of security or a founding engineer. If that's you, this is for you.
An AI audit, in practice, may be as simple as a prompt like “find ALL bugs in this repo. Make no mistakes!” and a code repo, and the report comes back in minutes or hours, instead of days or weeks. Set those two options side by side under a launch deadline, and the AI option does not look reckless. It looks modern, fast, and cheap enough to not think twice about. Not to mention the rumors that "nowadays all security auditors use LLMs." So then, why not just go for a simple AI scan for a super low cost rather than pay a large amount to an audit firm?
But not all "AI audits" are the same. Founders comparing quotes are often comparing three very different categories of solutions under the same label, and the gap between them is not always clear. I'll try to break it down below:
1. Fully-automated audits:
No humans in the loop at any stage. Quality varies a lot between providers, and I won't pretend these systems don't catch real, important bugs (they do). But no one who's actually used one can honestly claim the false positives and mislabeled severities aren't a problem. You get a report, but the extra cost for you is the triaging effort that needs to be done by your engineers. To make things worse, non-security engineers sometimes don't understand findings and incorrectly flag real issues as false positives.
2. Semi-automated audits:
A human checks the output at the end of an AI scan. This is what we run as our own Preaudit solution (read more about it here). It's a reasonable filter, but it is not a full audit. Similar to fully-automated audits, this system might miss bugs. This is acceptable for a bug bounty program, but it's not acceptable for a full audit. There's no such thing as 100% secure, and humans still miss bugs, but they also find bugs that AI misses, and you don't want an attacker to be the one who finds what your AI missed first.
3. Hybrid audits:
What most serious firms, including ours, actually deliver when they say "audit." A human in the loop looks like this: an auditor reads the code and builds real understanding of it, while running leads, ideas, and open questions by an LLM the whole way through. The AI speeds the auditor up. It doesn't replace them.
What I want to help you avoid is getting handed option 1, priced and timed like it competes with option 3.
Now, this is not an anti-AI argument. AI is useful in security work: it reduces ramp-up time on an unfamiliar codebase, it spots known vulnerability patterns fast, it writes edge-case tests a human would never get around to on a deadline. None of that is where the risk sits.
The risk sits in what AI cannot see due to lack of context. That context doesn't only live in the code, it lives in the product conversations and the kickoff meeting, the messy human context a reviewer builds by talking to the team. A human auditor gets a gut feeling something is off before locating the bug. Business logic bugs and multistep attack chains, the ones that drain treasuries, are still a human catch.
I'm not saying that projects should not use AI. What I mean is there is a time and a place where each of the 3 categories should be used:
-
Use fully-automated audits during the development process, ideally integrated into your CI pipeline, making sure you fix relevant findings before you do an audit. Launching without an audit is irresponsible IMO, but such a lightweight AI scan is the least every project needs to do.
-
If you're just launching the project for the first time and you don't have a 6+ figure TVL, and you don't know if there's even a good product-market fit for your project, then start with a semi-automated audit for this launch (especially if you really can't afford a full audit). However, one thing you need to be really disciplined about here is doing a full audit as soon as you hit 6 figures in TVL and get some traction, because that's when serious attackers will start looking at your project (and yes, even if it's closed source).
-
If you have a mature project with 6+ figures in TVL, then you should do a hybrid audit even if you make small changes. Keep in mind that I'm not saying you should audit the full codebase if you make a small change, that's what Continuous Security engagements are for.
Hope this helps. Ship Safely. 🚀
RELATED SERVICES