Reviewing AI-generated code: the checklist I use as a tech lead

Starting Monday, an agent opens a draft pull request on this blog every weekday: one post, plus the code the post is about. The rule is simple. Nothing it writes goes live until I've reviewed it.

I've been writing software with coding agents for a while. My open-source Claude Code plugin runs a small team of them in parallel. Before that I owned code review for a team of four engineers at KitchenSync, and those habits still apply. But reviewing code an agent wrote is not the same as reviewing a colleague's code. This is the checklist I use for it.

What's different about AI-generated code

It's cheap to produce, so review becomes the bottleneck. A person writes a few hundred lines a day. An agent can write that in minutes. The pressure to skim goes up, and so does the cost of skimming.

It looks finished. The naming is consistent, the comments are tidy, and the PR description sounds sure of itself. With a human, sloppy code is often a hint that something is wrong. Agent code doesn't give you that hint. Its mistakes are rarely typos. They're what I called plausible but wrong: code that reads well and does the wrong thing.

You can't ask it why. With a colleague, "why did you do it this way?" gets you the reasoning they actually had. An agent will give you a reasonable answer whether or not that's what drove the change. So the diff has to explain itself.

Before the review: limit what the agent can do

The cheapest review is the one you don't need, because the agent couldn't do the risky thing in the first place. The routine for this blog has written rules:

• It opens draft pull requests. It never merges, deploys or pushes to main.
• It never touches infra/. The Terraform for this site is mine to change.
• One task per pull request, on its own branch.
• It must run the code and quote only the numbers the run printed. No invented metrics.

The same idea shows up in my agent plugin: every agent declares which tools it can use and a maximum number of turns. I'd rather narrow what an agent can do up front than try to catch everything in review.

Let the machines go first

People shouldn't spend review time on things a machine can check, and that's even more true when the author is a machine. On this site the build fails if a post uses a category or tag that isn't in the allowed list, so an agent can't quietly make the taxonomy sprawl again. Types, tests and lint run on every change.

If CI isn't green, I don't start reading.

The checklist

1. Read the diff, not the description. The PR summary is the agent's opinion of what it did. The diff is what it actually did. I read the summary last, and only to check it against the diff.

2. Did it stay in scope? Look for files that have nothing to do with the task. Common ones: formatting changes across a whole file, a renamed helper "while I was there", an updated lockfile nobody asked for. Every unrelated line is something else I have to review, and it hides the real change.

3. Did the tests change so they'd pass? This is the one I check most carefully. When an agent can't make a test pass, it's tempted to change the test: weaker assertions, a skipped test, or a mock that replaces the very thing being tested. Any edit to an existing test gets a line-by-line read.

4. Does everything it imports actually exist? Agents invent functions, options and whole packages that sound right. Check every new dependency by hand. Is it the real package, a maintained one, and pinned? For APIs, check the version you actually run. When I wrote the LLM feature series, I type-checked the code against the exact library versions it uses, because code written against the wrong version looks perfectly fine until you run it.

5. Did it reinvent something you already have? An agent sees the files in its context, not the whole codebase. It will happily write a new date helper or validation function next to the one your shared library already has. At KitchenSync every service used the same shared TypeScript library. A second copy of a rule is how two reports end up disagreeing.

6. Where do the errors go? Look for catch blocks that log and carry on, and fallbacks like return [] or a default value that turns a failure into a silent wrong answer. Agents like code that doesn't crash. I prefer code that fails loudly where it should.

7. Did it cross a security boundary? I go through these one by one:

• Is input from users or other systems validated before it's used?
• Are there secrets in code, in config or in test fixtures?
• Is there an auth check on every new route, not just the first one?
• Does any new IAM policy use * where a specific action would do?

Agent code usually works on the happy path. These are the places where "works" isn't enough.

8. Is every claim backed by something that ran? This goes beyond numbers in a blog post. A comment that says "this is O(1)", a README that says "handles retries", or a PR description that says "all tests pass" are all claims. I check each one against a run, or delete it.

9. Is it small enough to review properly? The rule I used with people still applies: a big PR gets a quick "looks good", a small one gets a real review. If an agent hands me a change I can't review in one sitting, I ask it to split the work into steps, not for a summary.

10. Could I explain it to someone else? This is the final gate. If I can't explain why the change is correct, I don't merge it, however good it looks. When it breaks in production, nobody will ask the agent. They'll ask me.

The short version

• Limit what the agent can do before it starts.
• Let CI reject what a machine can catch.
• Read the diff, not the description.
• Be suspicious of scope creep, edited tests, new dependencies and quiet fallbacks.
• Check security boundaries one by one.
• Only merge what you could explain.

Agents change who writes the first draft. They don't change who is responsible for what ships. The daily routine starts on Monday, and in a few weeks I'll write about what this checklist actually caught.