Audit Methodology
Smart Contract Audits and AI Audits: What One Discipline Can Teach the Other

Two assurance disciplines emerged in the same decade, for the same reason: software started shipping into places where mistakes could not be withdrawn. Smart contract auditing grew up around code that holds money. AI auditing is growing up around models that give advice, make decisions and talk to millions of people.
They look unrelated. Read their methods side by side and the overlap is uncomfortable: both review an artefact nobody wrote by hand, both face adversaries rather than users, both know the review has to happen before release. Where they diverge is more instructive still, because each field is strong exactly where the other is weak.
This piece is written from the contract side. We build an automated smart contract audit, so our bias is visible, and we have an interest in claiming automation works. The honest version of that claim is narrower than the marketing version, and the comparison below is partly an argument for why.
Where the two disciplines agree
Both audit a system nobody fully understands
A deployed contract and a trained model share an awkward property: the artefact is not the source. Solidity compiles to bytecode that executes with semantics the author rarely reasons about directly. A model's behaviour lives in weights nobody wrote. In both fields the auditor examines something that was produced rather than authored, and in both the gap between intent and artefact is where the failures hide.
Both carry consequences the author cannot take back
Deployed EVM code is transparent, adversarially accessible, and hard to patch. A model that has shipped to millions of users cannot be recalled from the conversations it has already had. Neither field gets the ordinary software comfort of pushing a fix on Tuesday, and both have consequently developed the same reflex: the review happens before release, because afterwards is too late.
Both are judged by adversaries, not by users
A web application is mostly used by people trying to accomplish something. A smart contract holding funds is probed continuously by people trying to take them, and a deployed model is probed by people trying to make it do what it was built not to do. The threat model is not accidental misuse. It is a motivated attacker with time.
Both fields built red teams before they built standards
Smart contract security grew out of post-mortems, bug bounties and competitive audit platforms long before any written standard existed. AI evaluation is following the same path: jailbreak research, model cards and evaluation harnesses arrived well before anything resembling an audit profession. Practice preceded doctrine in both cases, which is why both still argue about what an audit is.
Where they part company
The differences are not matters of degree. They change what an audit can promise.
What counts as a failure
One has a ground truth, the other mostly does not
Smart contract
Deterministic. A reentrant withdrawal either drains the contract or it does not. Given the same state and the same transaction, the outcome is the same every time, which is why a finding can be reproduced and a fix can be proven to close it.
AI model
Statistical. A model that refuses a harmful request ninety-nine times in a hundred has not been fixed; it has been improved. There is no state from which the behaviour is guaranteed, so a finding is a rate, not a fact, and a fix shifts a distribution rather than closing a path.
Size of the attack surface
One has a bounded surface, the other has language
Smart contract
Finite and enumerable. A contract exposes a fixed set of functions with typed parameters. Large, sometimes combinatorially unpleasant, but countable — which is what makes static analysis and symbolic execution possible at all.
AI model
Unbounded. The input is every string a human or another model can produce, in every language, including ones the evaluators do not speak. No enumeration exists, so coverage can never be claimed, only sampled.
Cost of re-checking after a change
One can be re-verified cheaply, the other cannot
Smart contract
Minutes. Re-run the analysers against the new source and compare. This is precisely what makes automated contract auditing viable as a product.
AI model
Days and substantial compute. Every evaluation suite must be re-run, and a change that improves one behaviour routinely degrades another, so the whole battery matters rather than the part that was touched.
Nature of the harm
One loses money, the other loses something harder to price
Smart contract
Direct and measurable. Roughly 3.4 billion dollars were stolen across crypto in 2025, and the loss shows up in a wallet on a known date. That measurability is why insurance can be written against it at all.
AI model
Diffuse and contested. Harms are reputational, informational, discriminatory or societal, often spread across many people in small amounts, often disputed. There is no block explorer for a model that gave consistently worse advice to one group of users.
Evidence available to the auditor
One has a public record of every failure
Smart contract
Total. Every exploit is visible on-chain forever, with the exact transaction sequence. The field learns from a shared, verifiable corpus of real attacks, which is where detection rules come from.
AI model
Fragmentary. Most incidents are known only to the operator. Public evidence is mostly screenshots, which are trivially faked, and the model that produced them may no longer exist.
Maturity, measured honestly
Smart contract auditing is roughly a decade old and AI auditing is a few years old, so the obvious conclusion is that one is ahead. It is ahead on method and behind on consequences.
| Dimension | Smart contract | AI |
|---|---|---|
| A named profession Dozens of firms sell smart contract audits with comparable deliverables. AI auditing covers everything from bias testing to red teaming to compliance paperwork, often under the same word. | Yes, since roughly 2017 | Emerging, no stable definition |
| A maintained standard EthTrust defines graded levels with explicit requirements, and says which are machine-checkable. The NIST AI Risk Management Framework and ISO/IEC 42001 describe process and governance rather than what a model must do to pass. | Yes — EEA EthTrust Security Levels v3 | Partial — frameworks, not pass criteria |
| Reproducible findings A contract finding ships with the transaction sequence that triggers it. A model finding usually ships with a prompt that may not reproduce on the next version, or even the next sampling run. | Yes | Rarely |
| Automated tooling that practitioners trust Static analysers are standard equipment for contract reviewers. Automated model evaluation exists and is improving quickly, but no tool is yet treated as settling a question. | Yes, for part of the surface | Early |
| A liability regime This is the one axis where AI is ahead. The EU AI Act imposes obligations on high-risk systems with real penalties; smart contract audits are still governed mostly by the disclaimer at the back of the report. | Weak, mostly contractual | Forming, and faster |
What automation actually reaches
Both fields sell automation, and both risk overselling it the same way: by reporting what the tooling checked rather than what it could not. The split below is the one that matters, and it is roughly parallel in the two disciplines.
- Smart contract
- Known vulnerability classes with syntactic signatures: reentrancy, unchecked calls, access control, arithmetic, transaction ordering. EthTrust calls this level [S] and defines it as what static analysis verifies without human judgement.
- AI model
- Benchmarked capabilities and refusal rates against a fixed suite. Regression testing between versions. Detection of known jailbreak families.
- Smart contract
- Whether a flagged pattern is a deliberate design choice. A proxy's fallback delegatecall looks identical to an unprotected one; only someone who understands the architecture can tell them apart. EthTrust level [M].
- AI model
- Whether an output is actually harmful in context, which depends on who asked, why, and what they would have done otherwise. Mostly unresolved by any automated judge.
- Smart contract
- Business logic. Does the yield formula do what the whitepaper promised? Around 89% of 2025 DeFi protocol losses came from exploits specific to the protocol's own logic, which no static analyser sees. EthTrust level [Q].
- AI model
- Emergent behaviour nobody thought to test for, and failures that only appear at scale or over long interactions. The known-unknowns are the published benchmarks; the unknown-unknowns are what actually goes wrong.
The pattern repeats: automation handles the catalogue of known failures, struggles with whether a flagged pattern is deliberate, and does not reach the failures specific to what this particular system was supposed to do. In contract auditing that last band is where most of the money is lost. There is no reason to expect AI to be kinder.
What each field could borrow
What AI auditing could take from smart contract auditing
- Say what your audit does not cover, in writing. The most useful thing contract auditing produced is not a tool, it is the convention that a report states its scope and its limits explicitly. An AI evaluation that reports a pass rate without naming what was not tested invites a reading it cannot support.
- Build a shared, public corpus of real failures. Contract security advanced because every exploit is on-chain and anyone can study it. AI auditing has no equivalent, and incidents stay private, so each team rediscovers the same failure modes.
- Separate what a machine settles from what a human must judge. EthTrust's graded levels are a modest idea with large consequences: they let a buyer see exactly where automation stops. AI evaluation would benefit from the same honesty.
What smart contract auditing could take from AI auditing
- Treat findings as rates, not facts. Contract auditing speaks in binaries — vulnerable or not — and that binary hides confidence. AI evaluation has had to be statistical from the start, and its vocabulary for uncertainty is better.
- Take governance and operations seriously. Most 2025 crypto losses came from stolen keys and social engineering, not contract code. AI auditing inherited a process-and-governance tradition from safety engineering; contract auditing largely did not, and keeps reviewing the code while the losses happen beside it.
- Expect regulation and prepare for it. The EU AI Act gave AI auditing a reason to standardise its deliverables. Contract auditing has no equivalent pressure and has standardised less as a result, which shows in how differently two firms' reports read.
The uncomfortable conclusion
Smart contract auditing is the more mature discipline and still does not prevent most of the losses in its own field. The failures that cost the most in 2025 were stolen keys, social engineering, and logic specific to each protocol — none of which a code review catches. A decade of method produced excellent coverage of a category of bug that is no longer where the money goes.
That is the warning for AI auditing. It is possible to build rigorous, reproducible, well-tooled evaluation of a well-defined slice of behaviour, and for that slice to drift away from where the harm actually occurs. Benchmarks measure what is measurable, and the measurable keeps getting solved.
The defensible position in both fields is the same, and it is narrower than either sells: state precisely what was checked, state what was not, and make the boundary legible to someone who is not an expert. In contract auditing a standard now exists for drawing that line — the EEA EthTrust Security Levels separate what automated analysis settles from what needs a human. AI auditing does not have its equivalent yet. It will need one, and it would save time to copy the idea rather than rediscover it.
How Pépouze draws the line
Our audit is fully automated, and we say which level that reaches rather than implying it reaches all of them. We report on the classes a machine can settle from source, name the engines and versions that produced every finding, and state in the report itself what no automated pipeline examines.