Tinker AI
Read reviews
intermediate 4 min read AI-assisted

Measuring your team's AI-assist share

Published 2026-05-20 by Owner

A team that wants to know whether AI tools are helping should not measure the share of code an AI wrote. That number rewards accepting more autocompletes. Measure outcomes instead. This guide collects three of them with a PR-template checkbox, a label, and a short gh script — no new infrastructure.

Individual versus team questions

The individual version of this question — covered in measuring your own AI-code share — is about your own habits. The team version is different. You are not asking whether your team uses AI; in 2026 it does. You are asking whether your team uses it well, and “well” is an outcome, not an authorship percentage.

Three metrics worth collecting

  • Ship-without-rework rate — of AI-assisted PRs, the share that merge and are not substantially rewritten before release.
  • 14-day revert rate — of AI-assisted PRs that merged, the share reverted or hotfixed within two weeks.
  • Review-time delta — median review time on AI-assisted PRs against your non-assisted baseline.

A high assist share with a clean ship-without-rework rate is the good case. A high assist share with a climbing revert rate is the case people prefer not to look at, which is why you measure it on purpose.

Tag the PRs at the source

Add one line to the pull-request template so the author self-reports:

- [ ] This PR was substantially AI-assisted (agent, Composer, or Chat)

Turn that checkbox into a label with an Action so the data is queryable:

on:
  pull_request:
    types: [opened, edited]
jobs:
  label:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/github-script@v7
        with:
          script: |
            const body = context.payload.pull_request.body || "";
            const assisted = /\[x\] This PR was substantially AI-assisted/i.test(body);
            if (assisted) {
              await github.rest.issues.addLabels({
                ...context.repo,
                issue_number: context.payload.pull_request.number,
                labels: ["ai-assisted"]
              });
            }

Pull the numbers

Once PRs carry the label, query them with gh. Merged AI-assisted PRs over the last quarter:

gh pr list --label ai-assisted --state merged --search "merged:>=2026-02-20" --limit 200 --json number,mergedAt,title

Reverts that reference an assisted PR:

gh pr list --state merged --search "revert in:title merged:>=2026-02-20" --json number,title,mergedAt

Compare the counts to get the revert rate, and use the merge and review timestamps to compute the review-time delta against an unlabeled baseline pulled the same way.

Watch where the rework concentrates

The three rates above are team aggregates, and aggregates hide the thing you most want to find: whether the cost lands on specific people. Break the revert rate down by author. If less-experienced engineers ship assisted PRs that revert at several times the rate of seniors, the tool is not raising the floor — it is letting people merge code they cannot yet evaluate, and the senior review queue is absorbing the difference.

gh pr list --label ai-assisted --state merged --json author --jq '.[].author.login' | sort | uniq -c | sort -rn

Cross-reference the authors of reverted assisted PRs against that list. A flat distribution is healthy; a concentrated one is a training signal, not a tooling win.

Read it honestly each quarter

Run the pull once a quarter and resist the urge to celebrate a rising assist share on its own. The share going up is not a result; the revert rate staying flat while the share goes up is. If the revert rate climbs with the share, the tools are moving work from authoring into firefighting, and that is the signal the headline 51% is built to hide — see the 51% number and what “assisted” hides.

#metrics #workflow #teams