What a Human Still Has to Check in AI-Written Code

Why AI code changes what review is for

Human code review evolved to catch human failure: typos, forgotten branches, fatigue mistakes, the shortcut taken at 6pm. AI-written code fails differently. It is syntactically immaculate, stylistically consistent and complete-looking — and its defects hide at the level of intent. The code does exactly what it does, flawlessly; the question is whether that is what the business needed.

This is why review effort has to move up a level. Reading AI code for typos wastes the reviewer; the linter already did that. The human's irreplaceable job is checking the code's behaviour against reality: who is allowed to do this, what happens to the money, what happens when the input is empty, duplicated or hostile. Those questions need context the model was never given.

What review research says about human limits

The case for concentration is quantitative, and it predates AI. The largest published study of peer code review — SmartBear's work with a Cisco Systems team, spanning 2,500 reviews of 3.2 million lines of code over ten months — found reviewers are effective on 200 to 400 lines per session, that a 60–90 minute review of that size yields 70–90% defect discovery, and that pushing much past 450–500 lines an hour collapses defect detection in the large majority of cases.

Now put AI's output rate against those limits. An agent can generate several thousand lines in a working day; at an effective human rate of about 300 lines an hour, comprehensively reviewing one day of generation is a week of senior reading. The arithmetic does not merely favour concentrated review — it forbids the comprehensive kind. The eight-category checklist below is what survives that collision.

The quality data behind the checklist

Independent measurement backs the same categories. Veracode's 2025 GenAI Code Security Report, which tested more than 100 models across 80 curated tasks, found generated code introduced a security vulnerability in 45% of cases — and failed to defend against cross-site scripting in 86% of relevant samples. That is the direct case for the authorisation and security items on the list.

The maintainability items have data too. GitClear's analysis of over 200 million changed lines found duplicated code blocks rose eightfold during 2024, refactoring fell from about a quarter of changed lines in 2021 to under 10% in 2024, and copy-pasted lines exceeded moved lines for the first time in its dataset. And Stack Overflow's 2025 survey found 46% of developers distrust the accuracy of AI output against 33% who trust it — a professional consensus that verification, not faith, is the correct operating mode.

The eight things a human must check

This is the checklist as we run it. Every item earned its place through a defect that automation missed, and each is checked by a senior engineer on every release regardless of how clean the diff looks.

• Authorisation logic — not just 'is the user logged in' but 'may this user touch this record'; generated code passes the first and forgets the second more than anything else we catch.
• Money and quantity arithmetic — currency rounding, discount stacking, refund paths, timezone-sensitive billing dates; errors here are silent and compounding.
• Deletion and migration paths — anything that destroys or transforms data gets read line-by-line, because these are the mistakes you cannot patch after the fact.
• Error handling — hunt for catch blocks that log nothing and carry on; models generate soothing error-swallowing wrappers that convert loud failures into quiet data loss.
• Secrets and configuration — keys pasted into code, permissive CORS, debug endpoints left enabled, verbose errors leaking internals.
• Dependency provenance — every new package verified as real, maintained and licence-compatible; models occasionally invent plausible package names, which attackers exploit.
• Performance at real volumes — generated code loves loading everything then filtering in memory; fine with 50 demo rows, fatal with 50,000 real ones.
• Scope of change — diff the work against the brief and inspect anything the model did beyond it; unrequested 'improvements' are where surprises live.

What runs before a human looks

Human attention is the scarcest resource in an AI-assisted pipeline, so nothing reaches a reviewer until the machines are satisfied. The gate is boring and absolute: type checks and linters clean, dependency and vulnerability scans clean, and the full test suite green — with the tests themselves largely AI-drafted but human-curated, since a generated test that asserts the wrong behaviour just certifies the bug.

The ordering is the point. When automation runs first, the human arrives at a diff that is already syntactically sound and regression-free, and can spend their entire attention on the eight questions above. Reversing the order — humans first, machines later — burns senior hours on what a linter finds in milliseconds.

The process end to end

Assembled into a pipeline, here is what stands between generated code and your production system on every PINCLER release. Each layer exists because the layers around it have known blind spots.

Failure patterns that built this checklist

Concrete examples make the list less abstract, so here are three patterns — described generically — that reviews of generated code catch repeatedly. A booking flow where the confirmation endpoint checked that a booking ID existed but not that it belonged to the requesting user: working demo, open door. A retry wrapper around a payment call, added helpfully for reliability, with no idempotency key — under a flaky network it would have charged twice. An import script that filtered invalid rows silently, so a malformed spreadsheet would have vanished forty records without a trace.

Notice what unites them: every one passes a demo, a type check and most test suites, because each is correct code for a subtly wrong specification. That is the signature AI failure, and it is why the human step concentrates on intent rather than syntax. None of these took long to fix once seen — the entire value was in the seeing.

The arithmetic of concentrated review

Here is the cost model on a typical small build. Suppose a two-week project generates 12,000 lines of code, of which the genuine risk surface — authorisation, payments, deletion, migrations — is 1,800 lines. Concentrated review at 300 lines an hour is six senior hours; at $80 an hour, $480. Comprehensive line-by-line review of all 12,000 lines at the same careful rate would be 40 hours — a full senior week, roughly $3,200, wrapped around a build that might cost $1,800 in total.

That is the whole economic argument in two numbers: $480 buys expert eyes on every line that can lose money or leak data, while automation polices the remaining 10,200 lines for the failure classes machines catch reliably. It is also why 'we review everything by hand' from a vendor should prompt a follow-up question about arithmetic rather than a reassured nod — at honest reading speeds, either they are not, or their price says they are.

How deep to go: a risk-tiering guide

Review depth should be a decision, not a habit, and we tier it by blast radius: what is the worst thing this code can do if it is wrong? The tiers below are how that plays out in practice, and they are a reasonable default for any team adopting the checklist.

• Tier 1 — money, permissions, destruction: payment flows, authorisation, deletion, migrations. Line-by-line senior review, every time, with no exceptions for small diffs.
• Tier 2 — data handling and integrations: imports, webhooks, third-party API calls. Reviewed against the checklist with particular attention to the silent-failure and idempotency items.
• Tier 3 — presentation and content: pages, styling, copy changes. Automated gates plus a skim; human minutes here are taken from Tier 1's budget, so spend them sparingly.

How this plays out across our projects

Tiering is why rigorous review coexists with fast delivery rather than competing with it. Across PINCLER's 79 documented projects — the dataset is published at /research/what-you-can-build — the median delivery is 13 days with review inside the window, not bolted after it. Claude Code drafts on 63 of those builds and Cursor features on 75, which is to say nearly all the code starts life generated — and every Tier 1 path still gets the line-by-line treatment before anything ships.

The same discipline is what makes fixed pricing possible at all. Because the human hours are concentrated and predictable — six-ish on the risk surface of a typical build rather than forty across everything — the review cost can sit inside a $500–$2,500 quote instead of exploding it. Process discipline and price discipline turn out to be the same discipline.

What to do with this list

If you build with AI tools yourself, adopt the eight checks as a pre-merge ritual and let CI own the rest — it is a lightweight process that scales from solo founder to full team. If you hire a team instead, the list converts directly into due diligence: ask a prospective vendor which checks a human performs on every release and who that human is. A confident, specific answer tells you more than any portfolio.

This review pipeline is included in every PINCLER build — it is a large part of what a fixed $500–$2,500 price actually buys, alongside the written bug-fix warranty that backs it. If you want to see the end product of the process, browse our use cases for fixed scopes and prices, or book a free 30-minute call and we will walk you through a real review, checklist and all.