Do AI coding agents include tests often enough to trust a passing PR?

Do AI coding agents include tests often enough to trust a passing PR

No. A passing PR is not enough by itself, because AI agents include tests inconsistently and existing tests miss many changed lines. Treat green as a signal, then inspect coverage and behavior.

GitHub’s own pull request flow treats checks as one input, not the whole decision. Status checks show whether automated tests, builds, or other validations passed, and protected branches can require those checks before merge. That is useful, but it is a merge rule, not proof that the change is safe. A green PR can still be under-tested, especially when the diff is small, the tests are shallow, or the agent only touched implementation code and not assertions.

Recent research on agentic pull requests points in the same direction. In a 2026 study of 4,882 agent-generated PRs, agents added test changes in only 49.6% of PRs that modified code under test files. The same study found that existing tests covered only 61.5% of changed executable lines in Java and 27.0% in Python, and 64.8% of Python PRs had no changed line executed by any existing test. Those are not trust-building numbers, they are a warning that pass status often rests on incomplete coverage.

The part people get wrong is assuming "tests passed" means "the important behavior was exercised." It does not. A PR can pass because the tests do not reach the changed branch, because mocks absorb the real behavior, or because the change sits behind code paths that the suite never touches. Another study of autonomous agent PRs found that test-containing PRs became more common over time and had similar merge rates to non-test PRs, which sounds encouraging until you notice that merge rate is not the same as bug avoidance.

A passing PR deserves less trust when the agent changed application logic but did not add or update tests. In the coverage study, agent-written tests improved coverage in only a minority of Code + Tests PRs, 35.9% in Java and 22.5% in Python. The same paper found error-handling code was especially neglected, with miss rates reaching 86.0% in Java and 81.0% in Python. That is the inconvenient part: the code that breaks in production is often exactly the code that a green CI run leaves least examined.

A practical review step is simple: open the PR and check whether the test file changed, not only whether the workflow passed. If the agent added no test and the change is more than a trivial refactor, ask what behavior is actually protected. If the agent did add tests, ask whether they assert real outcomes or just mirror the implementation. A test that reproduces the code path without checking a meaningful invariant gives comfort, not confidence. GitHub’s review tools are designed for that exact inspection, including checking diffs and running local validation when needed.

The right question is not "did the PR pass?" It is "what evidence shows this change would fail if it were wrong?" For some changes, a passing PR is enough after a small spot check. For logic changes, boundary conditions, and bug fixes, you want explicit tests for the regression and at least one test that exercises the new behavior end to end. If the agent only touched implementation code, the burden shifts to the reviewer to prove the suite still protects the behavior the team cares about.

A concrete example is a payment or parsing bug fix. An agent can patch the branch, satisfy existing tests, and still leave the real bug unverified if the failing input is not in the suite. The safer review path is to reproduce the original failure, confirm the new test fails before the fix and passes after, and verify that the test does not only depend on mocks. That takes longer than reading a green check, but it is the difference between verification and decoration.

Use passing PRs as a filter, not as a verdict. The pass tells you the change did not break the checks you already had. It does not tell you the checks were broad enough, that the agent wrote the right tests, or that the risky branch was covered. In AI-generated code, that gap is common enough that a reviewer should assume it exists until the diff proves otherwise.

If you want the team rule in one sentence, use this: merge on green only when the PR also shows the behavior being protected. Otherwise, rerun the tests, add the missing assertion, or reject the change until the safety net is real. That standard is slower than blind trust, but it is cheaper than debugging a release that passed for the wrong reason.

If you are building a workflow around agent-written code, DevConnect keeps the exchange on owned work only, and it is free to use, including the closed test tracker, so you can organize reciprocal testing without paid shortcuts. The page at https://devconnectplatform.com explains the platform itself, but the review rule stays the same: a green PR is a checkpoint, not a guarantee.

FAQ

What should I check first on a green AI-generated PR Check whether the diff includes new or updated tests, whether the tests cover the changed behavior, and whether the CI run actually exercised the changed paths. A status check alone only proves the configured workflow passed.

Are agent-written tests usually enough on their own No. The research shows agent-written tests improve coverage in some cases, but only in a minority of code-plus-tests PRs, and important branches such as error handling are often missed.

Is a PR without test changes always bad No, not always. Documentation updates, mechanical refactors, and some low-risk cleanup work can be fine without test edits. The risk rises when the agent changes business logic, parsing, state transitions, or bug fixes that should have a regression test.

What is the fastest safe review habit for teams using coding agents Make the reviewer answer one question in the PR thread: which behavior would fail if this change were wrong. If that answer is unclear, add a test or a repro before merge.