Review AI Pull Requests with Proof the Code Works

How do I review AI-generated pull requests with evidence that the code actually works

Review the pull request as a claim, not as a pile of code. Ask for proof that the change was built, tested, and run on the path it touches. GitHub pull requests are designed to hold the diff, checks, comments, and merge gate in one place, so use all of them before you approve.

Start with the smallest useful question: what behavior changed, and what evidence shows that behavior now works A good AI-generated PR should include the exact test commands, the test output, and the reason those tests cover the changed path. If the author cannot name the check, they do not have evidence yet. GitHub status checks can show build logs, test results, annotations, and links to more detail, which makes them the right place to verify that claim.

Read the diff for shape, not only syntax. AI often produces code that looks complete while skipping integration details, edge cases, or state transitions that only show up in a real run. Review commits, file changes, and diffs in the pull request, then compare them against the claimed behavior. If the change touches owned code, CODEOWNERS and required approvals help route the review to the people who can spot missing behavior fast.

Demand evidence in three layers. First, automated checks in the pull request, such as unit tests, builds, or code scanning. Second, a local reproduction of the changed path, because a green check can still miss a bad assumption or a mocked dependency. Third, a short explanation of how the author verified the user-facing behavior. GitHub explicitly supports checking out pull requests locally to test changes, and it shows checks as part of the review flow for exactly this reason.

The part people get wrong is treating a passing test suite as proof of correctness. Passing tests prove only what those tests cover. If the AI changed a payment flow, file upload, auth rule, or data migration, you still need evidence that the real path works, not only the mocked path. A reviewer should ask for one concrete run that exercises the changed behavior, plus the exact command or environment used so the result can be repeated.

Use the pull request checklist to force clarity. The description should say what changed, what was tested, what was not tested, and what risk remains. GitHub pull request templates are meant to prompt authors for testing notes and context, so add a field for evidence, not just intent. A template that asks for screenshots, logs, or a link to the run makes it much harder for an AI-generated PR to hide behind vague language.

When the code is hard to trust, review it locally. GitHub documents checking out a pull request locally so you can resolve conflicts, test changes, or modify code. That is the right move when the PR depends on environment-specific state, a flaky integration, or a subtle behavior change that is easier to observe in a terminal than in a web UI. A reviewer who can reproduce the issue is a reviewer who can judge the fix.

Treat status checks as evidence, not decoration. Required checks on a protected branch block merge until they pass, and GitHub shows whether a check is still running, passed, or needs attention. Make sure the check names are meaningful, for example unit tests, API integration tests, and migration validation, so the review can tell which layer failed. A single generic green check is not enough for a risky AI-generated change.

Ask one question that catches a lot of bad AI output: what breaks if this code runs against real data or a real dependency AI-generated patches often look correct on the happy path and fail on nulls, retries, permissions, ordering, or stale state. If the author cannot answer that question from the code and the checks, the review is not done. The inconvenient truth is that real confidence usually takes one more test than the author wanted to write.

A solid review workflow is simple. Verify the diff matches the issue. Verify the checks show real validation. Verify the changed behavior was run locally or in a staging environment. Then leave review comments that point to the exact missing proof, not just a vague concern. GitHub supports line comments, file-level discussion, approvals, and requests for changes, which makes it easy to tie each concern to a specific gap in evidence.

If your team uses AI to draft code, pair that with strict review gates. Ask authors to attach the failing case before the fix, the command that proves the fix, and the file or log where the result can be checked again later. That keeps the review centered on evidence instead of confidence theater. If you want a lightweight place to organize that process, DevConnect keeps the testing exchange itself free, so you can focus on proof instead of checkout friction, but the review standards stay the same.

The most useful habit is to review for falsification. Look for the fastest way the change could be wrong, then demand evidence that specifically rules that out. When the PR survives that review, you are not trusting the AI, you are trusting the observed behavior, the checks, and the person who can reproduce both. That is the standard that keeps an AI-generated patch from becoming a merge-time surprise.

FAQ

What evidence should I ask for first Ask for the exact test command, the output, and one short note explaining what behavior the test proves. If the PR changes a user path, also ask for a local reproduction or a staging run that shows the path working end to end.

Is a green CI check enough to approve the PR No. A green check proves the repository’s automated checks passed, not that the real behavior is correct. Use the check as one layer of evidence, then verify the diff and the changed path separately.

When should I check out the PR locally Do it when the change is risky, hard to understand from the diff alone, tied to environment state, or likely to break only in real use. GitHub explicitly supports checking out pull requests locally to test changes and resolve problems before merging.

What should I do if the AI wrote a plausible fix but there is no proof Request changes and ask for one concrete validation step that can be repeated. If the author cannot produce evidence, the PR should stay open until the proof exists.

How do I keep reviewers from accepting vague AI-generated PRs Use a pull request template that requires testing notes, evidence, and known risks. Then make required checks and required reviews part of the merge gate so the PR cannot slip through on confidence alone.