Tenki’s startup program is live: up to $50K in credits and grants.Apply
Code Review

Each AI Agent Has a Distinct Bug Fingerprint

Eddie Wang
Eddie Wangengineering

Share Article:

Over a quarter of merged PRs in April 2026 were written end-to-end by AI agents. Not autocompleted. Not pair-programmed. Fully autonomous, ticket-to-PR code generation by Claude Code, OpenAI Codex, Cursor Background Agents, and Devin.

That's the headline from Greptile's "Rise of the Overnight Agents", published May 5, 2026. Daksh Gupta, Greptile's co-founder, studied millions of PRs across 65,000 organizations and found that agent-generated code is roughly on par with human code in overall quality. Revert rates are comparable. P0 flags per 10k lines of code are actually lower for most agents.

But here's the part that matters for anyone running code review at scale: each agent fails in its own specific, predictable way. Claude over-indexes on authorization bugs. Cursor produces N+1 queries at 3.45x the human rate. Codex trips on config and environment issues. The bugs don't disappear when you hand work to an agent. They just move.

And most review setups weren't built for that.

The data: 27.6% of PRs are now fully AI-generated

Greptile identified agent-authored PRs using three signals: bot authors on GitHub (like claude[bot]), Co-Authored-By footers in PR descriptions, and branch-name prefixes like codex/ or cursor/. In February 2025, only 0.86% of merged PRs showed evidence of full AI authorship. By April 2026, that number hit 27.6%.

That's not a rounding error. Roughly one in four PRs hitting your main branch was likely written without a human touching the keyboard.

The quality numbers are genuinely surprising. Codex and Claude both had lower revert rates than humans (1.19 and 1.80 per 1,000 merged PRs vs. 2.72 for humans). Codex's file churn sat at 5.7% vs. 10% for human code. And three of four agents flagged fewer P0 critical bugs per 10k lines than human-written PRs.

So the code is fine on average. The problem is in the tails.

Every agent has a fingerprint

Greptile ran keyword analysis across review comments tagged to each agent's PRs and computed how often each agent triggered a given bug category relative to the human baseline. The results aren't subtle.

Claude over-indexes on security and authorization issues. IDOR and missing tenant checks appear at 1.75x the human rate. Auth bypass sits at 1.50x. SQL injection at 1.50x. XSS at 1.57x. Claude also leaves stale comments and wrong documentation at 1.69x, which suggests it modifies behavior without updating the surrounding context.

Cursor Background Agents are the outlier. Almost every category sits above 2x the human rate. N+1 queries at 3.45x. Regression and breaking existing behavior at 2.37x. Off-by-one errors at 2.27x. Missing tests at 2.37x. Cursor BG was also the only agent that flagged more P0 critical bugs per 10k LOC than humans (0.145 vs. 0.099).

Codex is the steadiest performer overall, with the lowest revert rate (1.19 per 1,000) and lowest churn (5.7%). But its weak spots cluster around configuration: env var and config bugs at 1.35x, secrets in logs at 1.34x, and regressions at 1.34x. If you're shipping infrastructure or deployment code through Codex, those numbers matter.

Devin is at or below the human rate on every keyword category Greptile tested. Its security numbers are the best of the group (auth bypass 0.50x, IDOR 0.69x). But it has the highest revert rate at 3.50 per 1,000. Gupta notes that Devin's failures may not map to the keyword categories in his analysis; "completed the wrong task" wouldn't show up in a search for "off-by-one."

Why one-size-fits-all review rules miss this

Most code review configurations treat every PR the same. You define a set of rules, maybe enable some static analysis checks, and apply them uniformly. That worked fine when the failure distribution was roughly uniform too: human developers make a mix of all these mistake types, so broad coverage catches a reasonable share.

Agent-generated code changes that assumption. If Claude PRs are 1.75x more likely to have IDOR bugs than human PRs, but your review rules weight IDOR checks the same as everything else, you're under-covering the actual risk surface for a quarter of your PRs. The same applies to Cursor's N+1 problem or Codex's config issues.

The review checklists that engineering teams built in 2023 and 2024 were designed around human patterns. Nobody was writing "check for N+1 queries in every PR from Cursor" because that pattern didn't exist yet. Now it does, and teams that don't adapt their review layer are leaving predictable bugs uncovered.

Detecting agent authorship from GitHub metadata

Before you can route PRs into agent-specific review rulesets, you need to know which agent wrote the code. Greptile's methodology gives us a practical blueprint. There are three signals to check:

  1. Bot author on GitHub. PRs opened by devin-ai-integration[bot] or claude[bot] are the obvious case, but fewer than 1% of PRs use this pattern.
  2. Co-Authored-By footers. Claude and some other agents append a Co-Authored-By: Claude footer to the PR description when they open it. Nearly 20% of Greptile-reviewed PRs in March 2026 had these footers.
  3. Branch name prefixes. Codex names branches codex/<task-slug> (32k such PRs in March alone). Cursor's background agents tag branches with cursor/.

A review tool that reads these signals at PR open time can classify the authoring agent before any analysis runs. That classification becomes the key for selecting which rule profile to apply.

What agent-specific review rules look like

Based on Greptile's failure-pattern data, here's what a per-agent review configuration should prioritize.

For Claude PRs: authorization and tenant isolation

Claude's weakest categories are IDOR/tenant isolation (1.75x), off-by-one (1.64x), XSS (1.57x), auth bypass (1.50x), and SQL injection (1.50x). The pattern is consistent: Claude generates functional code that handles the happy path but misses authorization guard rails.

Review rules for Claude-authored PRs should flag any new API endpoint or data-access path that doesn't include an explicit tenant-scoping check. If a query touches user data without filtering by the authenticated user's organization or tenant ID, that's a review comment. Same for any route that accepts an ID parameter without verifying the requester owns that resource.

The stale-comment rate (1.69x) is worth noting separately. Claude often edits logic but leaves the original docstring or inline comment untouched. A rule that detects modified function bodies with unchanged JSDoc or docstrings catches this pattern reliably.

For Cursor BG PRs: query patterns and regressions

Cursor BG's numbers are the most dramatic in the dataset. N+1 queries at 3.45x, regressions at 2.37x, missing tests at 2.37x, off-by-one at 2.27x, timezone/date bugs at 2.09x, dead code at 2.05x.

The 3.45x N+1 rate means Cursor BG PRs touching ORM code should trigger mandatory query-plan analysis. Any new loop that issues a database query, or any eager-loading change that removes an include/join, is a candidate for review flagging. This is a specific enough pattern that a static check can catch most instances.

The regression number (2.37x) tells a different story. Cursor BG tends to solve the assigned task while breaking adjacent functionality. Review rules should expand the diff context window for Cursor PRs, checking whether modified files have callers or dependents that might be affected by the change. If a function signature changes, every call site should be verified.

For Codex PRs: config validation and secrets

Codex's failure profile is narrower. Env var/config bugs at 1.35x, secrets in logs at 1.34x, and regressions at 1.34x. Everything else clusters near or below the human baseline.

For Codex-authored PRs, review should verify that any new environment variable reference has a corresponding entry in the deployment configuration or .env template. Codex tends to add code that reads from process.env.SOMETHING without documenting the variable or ensuring it's set across environments. The secrets-in-logs pattern is similar: Codex logs debugging information that includes sensitive values. A rule that scans for logging statements containing variable names that match known secret patterns catches this.

Configuring Tenki's review layer for agent-specific rules

Tenki's Code Reviewer runs as a GitHub App that posts context-aware reviews on every PR. It already catches many of the bug categories Greptile identified, including IDOR violations, N+1 queries, and config issues. The gap for most teams isn't that Tenki can't find these bugs. It's that teams haven't configured their review rules to weight agent-specific patterns higher when the PR author is an AI agent.

The approach is straightforward. Tenki reviews PRs at the GitHub level, meaning it has access to the same metadata signals Greptile used for classification: the PR author, the branch name, and the PR body. When a PR arrives from a codex/ branch or carries a Co-Authored-By: Claude footer, the review can prioritize the checks that matter most for that specific agent.

At $0.50 per review, running Tenki on every PR is cheap enough that the per-agent configuration question isn't about cost. It's about signal-to-noise. If your team is merging hundreds of agent-generated PRs per week, tuning the review to catch the bugs that specific agent actually produces means fewer false positives on things it handles well and more attention on its blind spots.

The coverage gap is a configuration problem

Greptile's data makes a strong case that agent-generated code isn't worse than human code in aggregate. But aggregates hide the important details. A team that uses Claude Code heavily and doesn't scrutinize authorization on those PRs is carrying more IDOR risk than a team writing the same code by hand. A team shipping Cursor BG output without N+1 detection is accumulating query performance debt at 3.45x the normal rate.

The fix isn't to stop using agents. The 27.6% number isn't going down. The fix is to stop reviewing agent PRs the same way you review human PRs. Each agent has a fingerprint. Your review layer should know which one it's looking at.

Tags

#ai-code-review#claude-code#greptile#ai-code-quality#codex-cli

Recommended for you

What's next in your stack.