The CI Runner Is Where AI Review Gets Teeth

Every AI code review tool on the market delivers its verdict the same way: as a comment on your pull request. Copilot code review, CodeRabbit, Greptile, all of them. And every one of those comments comes with a button that says "Resolve" or "Dismiss."

Click it, and the finding is gone. No audit trail. No elevated permissions required. No friction at all. The developer who wrote the code is the same person who can make the review feedback disappear.

That's not a review gate. That's a suggestion box.

The dismissal problem is structural, not cultural

Teams don't ignore AI review findings because they're lazy. They ignore them because the tools make it effortless to do so, and because there's always a deadline.

GitHub's PR review UI treats AI-generated comments identically to human comments. Both can be resolved by the PR author. Both vanish from the default view once resolved. Neither blocks the merge button unless a separate branch protection rule explicitly requires a human approval.

That means AI code review, as currently implemented by every standalone tool, has the same enforcement power as a linter warning that nobody configured as an error. It's informational. You can read it, or you can scroll past it. The merge goes through either way.

This wouldn't be a crisis if the volume of AI-generated code were low. But it's not.

27% of merged code is AI-authored, and the review tools have no teeth

According to DX's Q1 2026 AI Impact Report, the average share of merged code authored by AI hit 27.4% across 500+ organizations. Among daily AI users, it's 30.8%. That number has been climbing steadily and shows no signs of plateauing in a meaningful way.

When a quarter of your codebase is being written by machines, the review layer isn't optional infrastructure. It's the last line of defense before that code hits production. And right now, every major AI review tool delivers that defense as a dismissible comment.

GitHub's new billing model makes the problem worse

On June 1, 2026, GitHub moved all Copilot plans to usage-based billing. Copilot code review now consumes GitHub Actions minutes in addition to AI Credits. Every review burns compute budget. Orgs can set user-level spending caps.

Think about the incentive structure this creates. Copilot code review is now a metered service. Each review costs the developer (or the org) real money. The review's output is a dismissible comment. So the rational behavior under budget pressure is: skip the review to save credits, or dismiss findings quickly to avoid triggering follow-up reviews.

The billing change didn't just fail to solve the enforcement gap. It created an active financial incentive to widen it.

How required status checks actually work

GitHub's branch protection rules include a feature called required status checks. When you configure one, a specific CI job must pass before a PR can be merged. If the job exits non-zero, the merge button is grayed out. You can't click through it. You can't resolve it. You can't dismiss it.

The only way past a failing required status check is an admin override. And that override does three things comment-based review dismissal doesn't:

1. It requires elevated permissions. Only users with admin or bypass permissions on the repository can do it. A regular contributor can't.
2. It creates an audit event. The override is logged in the branch protection audit log. You can see who bypassed which check, on which PR, and when.
3. It's deliberate. An admin override is an explicit decision to bypass a safety check. It's not a casual click on a "resolve" button between sips of coffee.

This is the same mechanism that enforces your test suite, your lint checks, and your security scans. Nobody argues those should be optional PR comments. So why is AI code review delivered differently?

The exit code is the review

This is the architectural distinction that matters. When an AI review tool runs as a CI job on a runner, the review outcome isn't a comment. It's the exit code of the process.

Exit 0: the review passed. The PR can merge. Exit 1: the review found critical issues. The PR is blocked.

This is how Tenki's code reviewer works. The review runs as a job on a GitHub Actions runner. If it finds issues above the configured severity threshold, the job fails. Because the job is wired as a required status check, the PR can't merge until the issues are addressed or an admin explicitly overrides.

The review still posts comments on the PR with specific findings, so developers get the same contextual feedback. But the enforcement happens at the infrastructure layer, not the UI layer. That's a fundamental difference.

The bypass escape hatch is a feature, not a bug

No review system should be completely unbendable. Hotfixes happen. False positives happen. There are legitimate reasons to merge a PR that a review tool flagged.

The question is how that bypass works. With comment-based tools, any contributor can silently resolve a finding. With a CI gate, the bypass requires admin permissions and leaves a permanent record in the audit log.

Both systems allow bypasses. Only one makes that bypass visible, accountable, and restricted to people with the authority to make that call. That's the difference between a gate and a suggestion.

For platform teams running compliance-sensitive codebases, this isn't an abstract concern. SOC 2, ISO 27001, and similar frameworks want evidence that code changes go through a defined review process. "We had an AI tool that posted comments, but developers could dismiss them" is a hard sell to an auditor. "We have a required CI check that blocks merges, and bypasses are logged" is a different conversation entirely.

Comment-based review was fine when humans wrote all the code

When AI-generated code was a novelty, the volume was low enough that human reviewers could catch what the tools missed. A developer might dismiss an AI comment, but another human would probably look at the PR anyway. The informal oversight compensated for the weak enforcement.

That model breaks down at 27% and climbing. The sheer volume of AI-generated PRs means human reviewers can't cover everything. They're already stretched thin. And under deadline pressure, a dismissible comment is the first thing that gets skipped.

Add GitHub's new metered billing to the mix, and you get a situation where the financial incentive and the path of least resistance both point toward less review, not more. The only way to counteract that is to put the review where developers can't route around it: in the CI pipeline, as a required check.

What to look for in an enforceable AI review setup

If you're evaluating AI code review tools and enforcement matters to your team, here's what separates a real gate from a glorified linter comment:

• Does the review run as a CI job? If the review outcome is only a PR comment, it can be dismissed. If it's a CI exit code, it can be enforced via required status checks.
• Can you configure severity thresholds? A good gate lets you define what constitutes a blocking finding versus an advisory one. You don't want to block merges over style nits.
• Is the bypass auditable? Admin overrides should generate audit log entries. If anyone can silently dismiss a finding, you don't have enforcement.
• Does it work with your existing branch protection? The tool should plug into GitHub's native required status checks or repository rulesets, not invent its own enforcement mechanism.

The comment-based AI review tools got the analysis part right. They can find real bugs. But they delivered the enforcement part as an afterthought, and GitHub's billing changes just made the gap more visible. The fix isn't better comments. It's putting the review in the one place on the platform where it can't be clicked away: the CI runner.