How AI code review works, and why most reviewers are noisy

In AI code review, a tool reads a pull request's diff and the code around it, asks a language model to find problems, filters what the model returns and posts the rest as comments on the pull request. Two of those steps decide whether developers keep reading its comments: the context the model gets, and what the tool drops before posting.

When Greptile launched its AI code reviewer, the top complaint was volume. On a pull request with 20 changes it could leave 10 comments, and authors stopped reading any of them. The team's audit found about 19% of the comments were good, 2% wrong and 79% nits: true, but nothing the author cared about.

Most AI reviewers start there, because a model can find something to say about any diff. Limiting it to what a busy engineer should read is the hard part, and it's where reviewers differ.

Tool facts come from each vendor's own docs and engineering posts, linked where they first appear. If you're new to the topic, start with our guide to AI code review.

This blog belongs to cubic (cubic.dev), an AI code review tool that appears in the benchmark table below, so weigh our view accordingly.

How AI code review works, step by step

How does AI code review work? Most reviewers run the same six steps:

1. A pull request triggers it. GitHub sends the reviewer a webhook event when a PR opens, gets new commits or leaves draft, and the reviewer fetches the diff, title, description and commits.

2. It drops files not worth reviewing. Copilot code review, for example, skips dependency manifests such as package.json and Gemfile.lock, log files and SVGs.

3. It gathers context: callers, definitions, history, linked tickets and team rules.

4. Models analyze the change, often in several passes or agents, each hunting a different kind of problem.

5. It filters the findings: verifies and scores them, removes duplicates and ranks the rest by severity.

6. It comments and learns. It posts the survivors as inline comments and folds reactions, replies and fixes into later reviews.

The pipeline most AI code reviewers share.

Steps 1, 2 and 6 are plumbing, and most tools do them the same way. Steps 3 and 5 decide quality: what the model gets to see, and what the reviewer lets through.

What an AI reviewer sees: the diff and its context

A diff is a narrow view: the changed lines, without the function that calls them or the config that makes them safe. Paste a diff into a chat model and that's all it has, which is the main limit of reviewing code with ChatGPT.

OpenAI measured what that costs. In December 2025 research, people graded review comments on recent commits from popular open-source repositories. Given only the diff, GPT-5 posted 2.74 comments per PR, and 17% were high-impact. Given the whole repository and the ability to run code, the same model posted 0.61 comments per PR, and 49% were high-impact. With the repository, the model said far less, and a larger share of it mattered.

Tools gather context in a few ways:

• Index the code ahead of time. Greptile builds a graph of every function, class and dependency when you connect a repository.

• Rebuild it for each review. CodeRabbit clones the repository into a sandbox, builds a dependency graph each time, and reads past PRs and linked Jira, Linear or GitHub issues.

• Fetch it on demand. Copilot code review calls tools to read related code and references as it goes.

• Read history and rules. One of the four agents in Anthropic's open-source code review plugin reads git blame. Most reviewers also read instruction files such as CLAUDE.md, AGENTS.md or .cursor/BUGBOT.md.

The kind of context matters. Atlassian's year-long study of its own reviewer, published in January 2026, found written review guidelines were the context that helped most, while PR descriptions and Jira tickets were too high-level to help much. And Uber's reviewer can't judge design because it doesn't see past PRs, feature flags, database schemas or docs.

How the analysis runs

Uber's 2025 write-up found that one prompt falls short even with strong models: its unfiltered output had hallucinated issues, duplicates and uneven quality. Most reviewers split the job instead.

• Specialists. Uber runs separate assistants for bugs, internal best practices and application security. Anthropic's managed Code Review sends several agents in parallel, each looking for a different class of issue.

• Repeated passes. The same model can give different answers on the same diff. The Bugbot that Cursor first launched ran eight passes over the diff in shuffled order and dropped any bug only one pass found.

• Static analysis as input. In October 2025, GitHub announced a public preview in which Copilot code review blends model findings with deterministic tools such as CodeQL and ESLint. CodeRabbit integrates more than 50 linters and security analysis tools, runs the relevant ones in a sandbox and has a verification agent check their results. Uber's rule of thumb: linters for patterns a rule can match, models for questions that need meaning, such as whether an integer holds a time.

Why most AI code reviewers are noisy

Wrong comments are the smaller problem, as Greptile's audit showed. Uber names two sources of noise: hallucinated issues, and valid issues that don't matter here, such as a performance concern in code that isn't performance-sensitive. Five things produce both.

Most of the diff is fine

A reviewer hunts for a few real problems among hundreds of fine lines. On Anthropic's own PRs, its reviewer found anything at all on 31% of PRs under 50 lines and 84% of those over 1,000 lines. OpenAI's reviewer commented on 36% of PRs that Codex wrote end to end. For a small PR, zero comments is often the correct review.

Say a 300-line PR has one real bug, and the reviewer is right about 99% of lines. It catches the bug and flags three fine lines along with it, so three of its four comments are wrong.

Hypothetical example: 300 lines, one bug, and a reviewer that's right about 99% of them.

Asked for problems, a model will produce some

Uber found a standalone prompt gave many false positives and many true but low-value comments. Greptile couldn't prompt its way out: no prompt cut nits without also cutting critical findings. OpenAI built a counterweight into Codex's open-source review prompt: return no findings unless the author would want them fixed.

Without context, the model guesses

Without the caller, a reviewer guesses how a function is used; without the config, whether a check is needed. The Codex prompt has a guard worth copying: a claim that a change breaks something elsewhere doesn't count until the reviewer can point to the code that breaks.

Nits crowd out bugs

Human reviewers write nits too. In Microsoft's 2013 study of 570 human review comments, only 14% were about defects, and code improvements led at 29%. Uber's engineers rated that mix when a bot wrote it: readability nits, logging tweaks, minor performance tweaks and style comments scored low, while correctness bugs and missing error handling scored high.

An ignored reviewer is worse than none

Google learned this from static analysis long before LLMs. It counts a warning as an effective false positive when the developer takes no useful action after seeing it, right or not. In the paper's words: "Developers, not tool authors, will determine and act on a tool's perceived false-positive rate." Google switched off code review analyzers whose Not useful rate passed 10% until their authors fixed them.

Noise costs time, too. At Beko in 2024, an AI reviewer built on Qodo's open-source PR-Agent averaged 3.65 comments per PR against 0.31 from humans. Developers resolved 73.8% of its comments, but average closure time rose from 5 hours 52 minutes to 8 hours 20 minutes, and developers complained of out-of-scope and irrelevant suggestions. Once authors learn to skim past a bot, they skim past the comment that mattered.

How good AI code reviewers cut the noise

The reviewers developers keep switched on stack several filters, and vendors describe them in their own docs and engineering posts:

Two cautions. Filters trade misses for quiet: AutoCommenter's first, high threshold held back predictions that were about 80% correct, and Cursor had to push its agentic Bugbot to flag more after it turned too cautious.

And a grader needs ground truth. In 2024 Greptile found that an LLM rating its own comments' severity was close to random. Teams that make graders work tune them on outcomes: Uber sets thresholds from developer feedback, and Atlassian trained its filter on which comments led to code changes.

What is agentic code review?

A code review agent fetches its own context. It reads the diff, decides what to look at next, calls a tool (search the repo, open a file, run a test) and repeats until it can back up a finding or drop it.

Cursor and GitHub both report gains from it. Cursor says Bugbot's biggest jump came when it switched to an agentic loop and the model pulled in the context it needed. In September 2026, GitHub gave Copilot code review shell tools to run builds, tests and scripts, and reported more high-severity findings and fewer nits.

Agents also fail in new ways:

• Cost and time. Anthropic's managed Code Review averages $15-25 and about 20 minutes per review, billed as usage on top of your plan (as of October 2026). Claude Code's local /code-review and /security-review commands review your branch or a PR in your own session instead.

• Running a stranger's code. A reviewer that runs tools on a PR runs whatever the PR contains. Security researchers showed this in practice in January 2025: a linter config inside a PR made a review service run attacker code and expose its secrets. The vendor switched the linter off and rotated the secrets the same day, and confirmed a fix that moved the linter into its sandbox six days later. Ask vendors what runs in the sandbox and what it can reach.

• Instructions hidden in the PR. The model reads titles, descriptions and code comments, so a PR can carry instructions. Anthropic's open-source security review Action isn't hardened against prompt injection, its README says, and OpenAI's Codex Action docs say to sanitize PR text.

• Too much trust. OpenAI warns that teams may treat a clean review as a guarantee of safety. A clean review only tells you what the reviewer didn't find.

How to measure an AI code reviewer

Count the comments developers act on. Reviewer teams keep landing on this metric: Google's effective false positives, Greptile's address rate, Cursor's resolution rate, Uber's share of comments addressed. Don't expect 100%. At Atlassian, human reviewers' comments led to a code change 44% of the time.

Martian's Code Review Bench runs this test across tools in public. Its online tracker samples merged open-source PRs where review bots commented, and an LLM matches what each bot suggested against what the developer changed afterward. It ranks tools by F1, which balances how many real issues a reviewer catches against how many of its comments are wrong. The default last-month view, checked October 10, 2026, across 17,505 scored PRs:

The board moves daily, and Martian cautions that its online data can't settle head-to-head comparisons: bots on the same PR see each other's output, and different kinds of repos pick different tools. It scores the whole product: the model plus the context, filters and feedback built around it. The top ten sit within 6.3 points of each other.

What this means when you pick a tool

Don't pick from a leaderboard alone. Run your top two candidates on a week of your own pull requests and mark every comment as fixed, ignored or wrong. Then check that the reviewer:

• stays quiet on small, low-risk PRs

• comments on lines the PR changed, and labels older problems as older

• shows how it checked each finding: the code path, a test run or the rule it applied

• lets you set severity, cap nits and skip paths from a file in the repo

• learns from thumbs-downs and replies without muting security findings

• tells you what runs in its sandbox and what a deep review costs

Pick the reviewer whose comments your team still reads a month from now. For the setup steps on GitHub, see how to set up AI pull request review.

Our ranked list of the best AI code review tools covers the 15 tools on Martian's board and who each one fits.

FAQ

What is a code review agent?

A code review agent is an AI reviewer that works in a loop instead of a single pass: it uses tools to search the repository, open files or run tests before deciding what to report.

Why do AI code reviewers have so many false positives?

Most of a diff is fine, and a model asked for problems will produce some anyway, often by guessing about code it never saw. Reviewers cut that noise with context, verification and team feedback; we've written more about the false positive problem.

Can AI code review replace human reviewers?

Not today. Uber's reviewer is much better at code-level bugs than at design, and Anthropic's own reviewer leaves approval to a human. Let AI take the first pass and keep people on intent, design and risk. Our guide to reviewing AI-generated code covers the human side.

cubic reviews pull requests on GitHub. If that's where your code lives, try it on a week of your own PRs next to your current reviewer: the trial lasts 7 days and needs no card.