# Measure AI coding cost per accepted change

> A repeatable worksheet for comparing model choices by accepted work, review time, retries, token categories, and provider charges.

Canonical HTML: https://canopyide.dev/guides/measure-ai-coding-cost-per-accepted-change
Article date: 2026-09-28

Token totals tell you how much a session consumed. They do not tell you whether its code was accepted, how much review it required, or what a subscription actually charged. Use one completed change as the unit of comparison.

## Define accepted before starting

Write two or three checks for the change: a behavior, a relevant automated check, and a review condition. For example, a settings form might save a name after refresh, reject an empty name, and leave no unresolved high-severity PR findings. If one run meets the checks and another does not, comparing their token spend alone is misleading.

## Capture the whole attempt

Record the implementation session, reviewer session, retries, and human review time under one task ID. Note the CLI, model, branch, starting commit, acceptance result, and exact commands run. When a model fails and a stronger model completes the job, count both sessions. Excluding failed first attempts makes a routing strategy appear cheaper than it was.

## Keep usage categories separate

If the CLI reports them, record uncached input, cached input, cache writes, and output separately for each model. Provider pricing treats these categories differently and can change. Our session-cost calculator lets you enter current rates and calculate an API-equivalent estimate. A plan quota or percentage is a different signal, and subscriptions, included usage, discounts, and reporting gaps can make the actual payment diverge from the estimate.

## Compare outcomes on similar tasks

For each strategy, calculate accepted changes divided by total attempts, estimated spend per accepted change, and human minutes per accepted change. If a task never reaches acceptance, record it as incomplete rather than assigning it zero cost. Compare several tasks of similar scope before concluding that a small-first or premium-first route helps. Different repositories and acceptance bars invalidate a simple ranking.

## Use the result to change one decision

Look for the expensive boundary: broad prompts, repeated discovery, context resets, review failures, or unnecessary premium-model turns. Change one part of the workflow and repeat the measurement. Keep quality visible by recording regressions or later fixes; a fast merge that needs rework next week was not a cheap completion.

## Copyable resources

### Copyable accepted-change worksheet

One row per session, then summarize all sessions for the task.

````text
Task ID and repository: [ ]
Starting commit and branch: [ ]
Acceptance checks: [ ]
Session 1: CLI/model [ ]; role [implement/review/fix]; outcome [ ]; tokens by category [ ]; estimated cost [ ]
Session 2 or retry: [same fields]
Human review minutes: [ ]
Provider charge attributable to task, if knowable: [unknown or amount with method]
Final state: [accepted / incomplete / reverted]
Later rework: [none / issue and time]
Compare only with tasks of similar scope and the same acceptance bar.
````

## Frequently asked questions

### Can I divide my subscription fee by the number of PRs?

You can use that as a rough monthly allocation if you label the method, but it is not the provider's per-PR charge. Keep subscription payments separate from API-equivalent token estimates.

### What if a CLI does not report cache tokens?

Mark the category unknown. Do not silently treat it as zero; the estimate may be incomplete.

## Sources and further reading

- [OpenAI API pricing and token categories](https://developers.openai.com/api/docs/pricing)
- [Claude API pricing and cache categories](https://platform.claude.com/docs/en/about-claude/pricing)
- [Canopy app README: usage and agent workspace](https://github.com/FluidWorksApp/canopy-ide/blob/main/README.md)

## Related Canopy pages

- [Your AI coding cost dashboard is not your bill](https://canopyide.dev/blog/your-ai-coding-cost-dashboard-is-not-your-bill.md)
- [How to reduce coding-agent token usage without losing the result](https://canopyide.dev/guides/reduce-coding-agent-token-usage.md)
- [Should I use a cheaper AI model first, then a stronger model to review?](https://canopyide.dev/use-cases/small-model-first-pass-strong-model-review.md)
- [How to evaluate an AI coding workspace on a real project](https://canopyide.dev/guides/evaluate-ai-coding-workspace-on-real-project.md)
- [How to stop an endless AI code review loop](https://canopyide.dev/guides/stop-endless-ai-code-review-loop.md)

Canopy runs installed coding CLIs; CLI accounts, model selection, and provider billing remain separate. Check the installed release before relying on version-specific behavior.