An implementer agent fixes a PR, a reviewer agent finds another issue, and the cycle repeats. Some new findings are real; others are questions, weak hypotheses, or work for a later change. A review loop needs an acceptance boundary and an owner who can decide. This guide uses a settings-form PR to show how to keep legitimate defects visible without making every new suggestion a reason to rewrite the feature.
Pin the PR and the decision it needs
Before another review, write down the issue, the PR head commit, and the smallest behavior that would make the change acceptable. For the example settings form: a signed-in user can save a valid display name; an invalid value shows a useful error without saving; an API failure leaves the form retryable; another user's settings cannot be changed. Name the tests and live checks that exercise those paths. A reviewer can still discover a serious defect outside this list, but it must explain the reachable failure and why this PR creates or exposes it. A vague instruction to 'find anything wrong' has no natural stopping point.
- Review the same head commit that the implementer says is ready.
- Keep the issue, acceptance checks, and final diff in the handoff to every reviewer.
- Separate release blockers from cleanup ideas before asking for fixes.
Turn each finding into a testable claim
Ask the reviewer to report the affected file and line, a path through the code, the user-visible or security consequence, and evidence or a reproduction. Open the cited code yourself. If the report says 'retry is broken,' force the API to fail, press Retry, and observe whether a second request is sent. If it says another user can edit the record, test the server boundary with two disposable accounts. A finding may be valid even if the agent could not run the app, but then it is a hypothesis to verify, not an established failure. GitHub's responsible-use guidance for AI review explicitly warns that reviews can miss problems and can raise false positives.
- If the concern cannot be tied to this diff or a reachable path, ask one clarifying question before editing.
- Record the outcome as confirmed, disproven, needs evidence, or outside the PR's scope.
- Treat a passing test as evidence for the case it covers, not a blanket approval of the feature.
Decide what happens to the finding
Use impact and relation to the PR to choose the next action. A cross-user write bypass is a blocker even if it surfaced on a late pass. A broken retry required by the issue is a blocker. A formatting preference or unrelated refactor may be useful but should not silently expand this PR. GitHub advises reading each review comment's intent and tracking out-of-scope feedback in a linked issue rather than enlarging the change. A reviewer can disagree with a decision; the PR owner should state the reason and leave the record visible.
| Finding | Evidence to request | Decision for this PR |
|---|---|---|
| Other user's settings can be saved | Cross-user request and server authorization path | Block merge; fix and retest denial |
| Retry button stays disabled after a 500 | Reproduction and second-request network trace | Block if retry is part of acceptance |
| Button label could be shorter | Design requirement or usability evidence | Decide with product owner; do not auto-loop |
| Unrelated profile page has old spacing | Separate issue and affected route | Track separately unless this diff caused it |
Run a bounded correction round
Send the implementer only confirmed blockers and the evidence behind them. Ask for a narrow fix and the test that would have caught the failure. Once the fix lands, verify the new head commit, run the failed case again, and inspect the entire latest diff for accidental changes. Re-review affected code and any newly changed boundary. There is no magic number of review passes: stop the automatic back-and-forth when the same finding recurs without new evidence, when reviewer and implementer disagree on intended behavior, or when fixes keep broadening the diff. Escalate that specific decision to a human owner who can clarify requirements or split the PR.
- A new commit can invalidate an earlier review or passing check; tie evidence to the latest head.
- Do not have two agents edit the same branch while one is still reviewing its previous state.
- If another serious defect appears, investigate it; a stop rule never overrides a reproduced blocker.
Close the loop with an explicit merge decision
The PR owner checks the current branch, required human approvals, unresolved conversations, latest CI status, and the running behavior. Resolve a review thread only when its concern has been fixed or a reasoned decision is recorded; GitHub's review guidance distinguishes a bug, question, requested approach, and suggested edit. Ask for a fresh review after substantial changes when the team's rules require it. A 'no new findings' AI message is not the finish line; the finish line is an accepted change with known remaining work assigned outside the PR. Canopy's review task can stage draft findings for a person to vet, while the branch, diff, and running result supply the evidence for that decision.
- Record which findings were fixed, declined with reason, or moved to linked issues.
- Record the exact commit, checks, and user journey used for acceptance.
- Measure retries and review time along with token use if you later compare review workflows.
Copyable resources
Bounded reviewer request
Use the exact PR head and remove private user data from reproductions.
PR / issue / latest head commit: [ ]
Acceptance checks and important failure paths: [ ]
Review read-only. Do not edit, post comments, approve, or merge.
For each possible blocker: cite file and line, reachable path, consequence, and reproduction or missing evidence.
Separate confirmed defects, hypotheses, questions, and out-of-scope improvements.
State what changed since the previous reviewed commit, if this is a second pass.
If no blocker is supported, say which paths you actually inspected and what remains untested.
PR owner triage: fix now [ ]; clarify [ ]; linked issue [ ]; decline with reason [ ].
After fix: latest head [ ]; failed case retested [ ]; final diff [ ]; CI [ ]; human decision [ ].Frequently asked questions
Why does an AI reviewer find new issues after every fix?
Each edit changes the review surface, and broad review prompts can surface different hypotheses on each pass. Some findings are real. Keep the issue, head commit, acceptance checks, and evidence standard stable so each new claim can be judged.
Should I stop reviewing after two passes?
No fixed pass count can make an unsafe change safe. Pause an unproductive loop when findings repeat without evidence or the agents disagree on intent, then have an owner decide. A reproduced blocker still needs a fix and retest.
Can I close an AI review comment because CI is green?
Only if the concern is actually resolved or a reasoned decision is recorded. CI may not exercise the reported path. Check the latest diff and reproduce the relevant case.
What should I do with a valid but unrelated finding?
If this PR did not cause it and it does not block the change under your team's risk rules, open a linked issue with the evidence and owner. Do not silently drop it or automatically widen this PR.