# AI Code Review Benchmark 2026 — Tenki vs CodeRabbit, Greptile, Cursor, Copilot (https://tenki.cloud/benchmarks/code-reviewer)

> For the complete documentation index, see [llms.txt](https://tenki.cloud/llms.txt)

Independent 2026 benchmark of AI code reviewers: 7 tools review the same 50 bug-introducing pull requests (122 real bugs) from five open-source repositories, graded per finding by a 3-judge LLM panel. Last updated: May 20, 2026.

## TL;DR

**Tenki is the #1 reviewer based on finding-level scoring.** In this independent 2026 benchmark of 6 leading AI code review tools, Tenki catches 84 of 122 real, production bugs across 50 pull requests from cal.com, Sentry, Grafana, Keycloak, and Discourse, graded per finding by a 3-judge LLM panel. That's 1.9× the next-best reviewer.

| Stat             | Value  |
| ---------------- | ------ |
| Real Bugs Caught | 84/122 |
| Catch Rate       | 68.9%  |
| LLM Judges       | 3      |
| Tools            | 7      |
| Repositories     | 5      |
| Code Reviews     | 350    |

## Methodology

Every pull request contains a real, merged bug-fix from an open-source codebase. We replay the bug-introducing diff into a clean fork and let each tool review it with its default configuration. Reviews are then graded by a 3-LLM judge panel using majority vote. No synthetic bugs, no repo-specific tuning.

- **Sources — 50 bug-fix PRs from 5 open-source repositories:** real merged fixes from cal.com (TypeScript), sentry (Python), grafana (Go), keycloak (Java), and discourse (Ruby). All five major server-side languages are represented in the bug set.
- **Replay — real bugs reintroduced, not invented:** the pre-fix diff is replayed against every tool with default settings: no custom rules, no repo-specific tuning, and full repository context for every reviewer. Every tool sees the same code at the same point in history.
- **Scoring — per-finding 3-judge LLM majority vote:** bugs are scored individually, not per-PR (which would over-credit drive-by comments). A bug counts as caught only if a line-level comment pinpoints the faulty code and explains its impact, and at least two of three independent LLM judges agree.

Test sources: [Cal.com](https://github.com/calcom/cal.com) (TypeScript, open source scheduling infrastructure), [Sentry](https://github.com/getsentry/sentry) (Python, error tracking & performance monitoring), [Grafana](https://github.com/grafana/grafana) (Go, monitoring & observability platform), [Keycloak](https://github.com/keycloak/keycloak) (Java, identity & access management), [Discourse](https://github.com/discourse/discourse) (Ruby, community discussion platform).

## Overall performance

**Tenki leads recall and F1 across every real bug.** Each individual bug is scored, not each pull request. Data is sorted by F1 and tools were kept at their default configurations. Higher-precision tools post fewer comments overall; Tenki's higher comment volume drives both higher recall and lower precision.

Dedicated PR reviewers:

| Tool       | Recall | Precision | F1   | Bugs caught |
| ---------- | ------ | --------- | ---- | ----------- |
| Tenki      | 68.9%  | 29.9%     | 41.7 | 84/122      |
| CodeRabbit | 28.7%  | 25.0%     | 26.7 | 35/122      |
| Greptile   | 36.1%  | 15.9%     | 22.1 | 44/122      |
| Copilot    | 24.6%  | 18.9%     | 21.4 | 30/122      |
| Graphite   | 3.3%   | 50.0%     | 6.2  | 4/122       |

Coding agents (bundled into the IDE; they review their own work in-loop, not post-hoc on a PR diff — shown for reference):

| Tool   | Recall | Precision | F1   | Bugs caught |
| ------ | ------ | --------- | ---- | ----------- |
| Devin  | 36.1%  | 47.3%     | 40.9 | 44/122      |
| Cursor | 32.0%  | 51.3%     | 39.4 | 39/122      |

## By severity

Do AI code reviewers catch the bugs that actually matter? Catch rate broken down by the severity of the individual finding. Critical bugs cause outages, data loss, or auth bypass. High-severity bugs break major user-facing flows. Medium bugs degrade behavior without breaking it.

| Tool       | Critical (7) | High (53)   | Medium (52) | Low (10)   |
| ---------- | ------------ | ----------- | ----------- | ---------- |
| Tenki      | 71% (5/7)    | 74% (39/53) | 69% (36/52) | 50% (5/10) |
| CodeRabbit | 43% (3/7)    | 25% (13/53) | 33% (17/52) | 20% (2/10) |
| Copilot    | 43% (3/7)    | 25% (13/53) | 23% (12/52) | 20% (2/10) |
| Cursor     | 14% (1/7)    | 34% (18/53) | 35% (18/52) | 20% (2/10) |
| Devin      | 29% (2/7)    | 45% (24/53) | 29% (15/52) | 30% (3/10) |
| Graphite   | 0% (0/7)     | 2% (1/53)   | 4% (2/52)   | 10% (1/10) |
| Greptile   | 71% (5/7)    | 32% (17/53) | 37% (19/52) | 30% (3/10) |

## By repository

How each reviewer performs across five production codebases. Per-repository recall: the share of real bugs each AI code reviewer caught in each codebase.

| Tool       | Cal.com (33) | Sentry (31) | Grafana (21) | Keycloak (15) | Discourse (22) |
| ---------- | ------------ | ----------- | ------------ | ------------- | -------------- |
| Tenki      | 91%          | 55%         | 52%          | 67%           | 77%            |
| CodeRabbit | 21%          | 16%         | 43%          | 40%           | 36%            |
| Copilot    | 21%          | 6%          | 33%          | 47%           | 32%            |
| Cursor     | 30%          | 13%         | 52%          | 47%           | 32%            |
| Devin      | 33%          | 29%         | 67%          | 27%           | 27%            |
| Graphite   | 0%           | 0%          | 14%          | 0%            | 5%             |
| Greptile   | 30%          | 13%         | 52%          | 53%           | 50%            |

## Case library

**Every real bug, every reviewer verdict.** An interactive table with one row per real bug — 122 findings across 50 production pull requests, filterable by severity and repository. Each cell shows whether that reviewer flagged that specific defect (Caught/Missed), decided by 3-LLM majority vote, with judge vote counts and a link to the actual review on GitHub.

## FAQ

### What does the Tenki AI code review benchmark measure?

It measures how reliably AI code review tools catch real, production-grade bugs in pull requests. Each tool reviews the same 50 bug-introducing PRs from five major open-source repositories, and three LLM judges decide by majority vote whether a tool's comment correctly identified each bug.

### Which AI code reviewers are compared?

Seven tools are compared: Tenki, CodeRabbit, Greptile, Graphite, GitHub Copilot, Cursor, and Devin. Tenki, CodeRabbit, Greptile, Graphite, and Copilot are dedicated PR review services. Cursor and Devin are coding agents shown for reference; they review their own work rather than third-party diffs.

### How is a bug counted as 'caught'?

A finding is caught only when a tool posts a line-level comment that explicitly identifies the faulty code and explains its impact. Three independent LLM judges vote per finding; majority rules. Drive-by comments that mention the file but miss the actual defect do not count.

### Which repositories and languages are covered?

Fifty pull requests are sampled from five large open-source repositories: cal.com (TypeScript), Sentry (Python), Grafana (Go), Keycloak (Java), and Discourse (Ruby). All five languages are represented in the bug set.

### How often is this benchmark updated?

We re-run the full benchmark when new tools launch, existing tools ship major model updates, or our bug corpus grows. The current run was published April 2026. Subscribe to the Tenki changelog to get notified when new results drop.

### Are the bugs real or synthetic?

Every bug is a real, merged fix from the open-source codebase. We replay the bug-introducing diff against each tool with default settings: no synthetic injections, no custom rules, no repo-specific tuning. Full repository context is granted to every reviewer.