AI code review: what it catches, what it misses, and how to set it up

AI code review: what it catches, what it misses, and how to set it up

AI code review uses a language model to read a pull request and the code around it, then leave comments on likely bugs, security problems and rule violations before a human reviewer looks. This guide covers what AI reviewers are good and bad at, how to set up GitHub Copilot and Claude Code reviews, review instruction files, prompts and a team policy.

In shortTL;DR
  • AI reviewers are good at local bugs, missed edge cases, unsafe patterns and rule violations in the diff; they are weak at product intent, architecture and anything outside the code they can see.
  • Treat an AI review as a first pass that runs on every pull request, never as the approval that lets code merge.
  • GitHub Copilot code review and Claude Code Review both post inline comments and do not block merges by default; Claude's /code-review command reviews a branch locally on any plan.
  • Steer the reviewer with a committed instruction file (.github/copilot-instructions.md, REVIEW.md or CLAUDE.md) that says what matters, what to skip and how severe each class of issue is.
  • Measure usefulness by the share of comments that lead to a code change, and tune the instructions until noise drops.

What is AI code review?

AI code review is the practice of having a language model read a pull request, along with the files around the change, and leave comments on likely bugs, security issues and violations of your team's rules. It runs in seconds to minutes, on every pull request, before a human reviewer opens the diff.

Modern AI reviewers are agents rather than single prompts. GitHub's Copilot code review can gather full repository context. Anthropic's Claude Code Review sends several agents over the diff and surrounding code in parallel, then runs a verification step that filters out false positives before anything is posted. Both post inline comments on the lines concerned and leave the merge decision to people.

The useful mental model: an AI reviewer is a fast, tireless colleague who reads every line, knows common bug patterns well, and knows nothing about why your team made the decisions it made unless you write them down.

What does AI code review catch, and what does it miss?

AI reviewers are strong on problems visible in the code and weak on problems that live in people's heads, other systems or production data. Knowing the split tells you where to point human attention.

AI reviewers tend to catchAI reviewers tend to miss
Off-by-one errors, null handling, wrong conditionsWhether the feature solves the right problem
Missing error handling and unhandled promise rejectionsArchitectural fit and long-term maintainability
Unscoped database queries and missing authorization checksBusiness rules that aren't written down anywhere
Secrets, injection patterns, unsafe deserializationPerformance under real production load
Inconsistency with nearby code and stated team rulesInteractions with services outside the repository
Tests that don't test the changeMigration and rollout risk across deployments
Typos in identifiers, dead code, leftover debug loggingSubtle concurrency bugs that need runtime evidence

Two more limits matter in practice. First, Claude Code Review works from reading the diff and the surrounding code without building or running anything, so a bug that only shows up at runtime may slip through. Second, Copilot code review skips some files entirely: according to GitHub, dependency manifests and lockfiles, log files and SVGs are excluded.

Note

Both vendors say plainly that their reviewers can miss bugs and security issues. A review with no findings tells you the reviewer found nothing, not that the code is safe.

For a broader checklist on AI-written code itself, read is AI-generated code safe.

How do you set up AI code review on GitHub?

You have four practical options: a managed reviewer from GitHub or Anthropic, a local review command, or a reviewer you run in your own CI. Pick based on your plan, how much control you want and who pays.

OptionStarts whenRuns onWho can use it
Copilot code reviewYou add Copilot as a reviewer, or automatically via settings or rulesetsGitHubCopilot Pro, Pro+, Max, Business, Enterprise
Claude Code ReviewPR opened, every push, or @claude review commentAnthropic infrastructureClaude Team and Enterprise organizations
/code-review in Claude CodeYou run it in your sessionYour machineAny Claude Code plan
Claude in GitHub ActionsYour workflow eventsYour CI runnersAnyone with an API key or cloud credentials

Copilot code review

  1. Open a pull request on GitHub.
  2. In the Reviewers section, select Copilot. You can choose a review effort of Lite or Balanced.
  3. Copilot leaves inline comments with suggested changes you can apply in a few clicks.
  4. To review every pull request automatically, enable it in your personal Copilot settings or through a repository or organization ruleset.

By default, Copilot's reviews don't count toward required approvals, and Copilot reviews a pull request once unless it is configured to review each push. At the time of writing (October 2026), GitHub's docs say usage is measured in AI credits, roughly $0.05 to $1 for a Lite review and $0.25 to $5 for a Balanced one, with Actions minutes billed separately.

Claude Code Review

  1. An Owner of your Claude organization follows the setup guide: connect GitHub, install the Claude GitHub App and pick repositories.
  2. Choose a review level: Quick, Standard (the default) or Deep. Deeper reviews cost more.
  3. Set when reviews run per repository: once when a pull request opens, after every push, or only on request.
  4. Anyone with write access can comment @claude review on a pull request to start one.

Each review posts inline findings ranked by severity, a summary and a check run named Claude Code Review. The check run never concludes as a failure, so it doesn't block merging. Inside a review, repository access is read-only and no commits are pushed.

Local review with /code-review

If you already use Claude Code, the /code-review command reviews your branch before you even open a pull request. It works on any plan and counts toward your normal usage.

Claude Code
/code-review                  # current branch plus uncommitted changes/code-review 1234             # a pull request by number/code-review main...feature   # one branch against another/code-review src/auth         # a directory/code-review high --fix       # higher effort, then apply the fixes/code-review --comment 1234   # post findings as inline PR comments

A low effort level reports only the findings Claude is most confident in, which means fewer false positives; higher levels broaden coverage. The review runs as a background subagent with its own context, so it doesn't fill your conversation.

Your own reviewer in CI

Running Claude in GitHub Actions gives you full control over the prompt, the model, when it runs and what it may touch, at the cost of maintaining a workflow file and paying API usage. Our guide to Claude Code GitHub Actions walks through a review workflow step by step.

How do you tell an AI reviewer what matters?

Commit an instruction file that defines severity, lists what to skip and adds checks specific to your codebase. Untuned reviewers default to generic advice, and generic advice is where most of the noise comes from.

Each tool reads its own file:

ToolInstruction files
Copilot code review.github/copilot-instructions.md, path-specific .github/instructions/**/*.instructions.md, AGENTS.md
Claude Code ReviewREVIEW.md (review-only rules) and CLAUDE.md (project context)
/code-review commandCLAUDE.md only

Claude Code Review reads REVIEW.md from the base or default branch, never from the pull request's own changes, so a pull request can't rewrite its own review rules. Anthropic's docs recommend protecting that branch for the same reason. A starting point adapted from the Customize Code Review docs:

REVIEW.md
## What Important means hereReserve Important for findings that would break behavior, leak data,or block a rollback: incorrect logic, unscoped database queries,PII in logs, and migrations that aren't backward compatible.Style and naming suggestions are Nit at most.## Cap the nitsReport at most five Nits per review.## Do not report- Anything CI already enforces: lint, formatting, type errors- Generated files under `src/gen/` and any lockfile## Always check- New API routes have an integration test- Database queries are scoped to the caller's tenant- Behavior claims cite a file and line, not an inference from naming

The last line is worth copying into any reviewer's instructions. Requiring evidence cuts false positives that cost authors a round trip. For what to put in project-wide context files, see CLAUDE.md best practices.

What prompt should you use for an AI code review?

When you ask an AI to review code yourself, in a chat or a custom CI job, give it the goal of the change, the risk areas and a strict output format. "Review this" produces a list of style opinions.

prompt
You are reviewing a pull request for a multi-tenant billing service.Goal of the change: add proration when a customer upgrades mid-cycle.Review the diff and any files it touches. Report only:1. Bugs that would produce wrong results or errors in production.2. Security issues: missing authorization, tenant leaks, secrets, injection.3. Missing or ineffective tests for the new behavior.For each finding give: file:line, severity (high/medium/low), whatbreaks, a concrete input that triggers it, and a suggested fix.Do not comment on formatting, naming or style.If you're not sure a finding is real, say what you'd need to check.If you find nothing, say so in one line.

Why this shape works:

  • The goal lets the reviewer notice code that does something other than what was intended.
  • Narrow categories stop it from padding the review with preferences.
  • A triggering input forces it to reason concretely, which exposes false positives.
  • Permission to find nothing reduces invented issues.
Warning

An AI reviewer reads untrusted text in pull requests: code comments, descriptions and docs written by whoever opened the PR. On public repositories, treat that as a prompt injection surface and never give a review job write access or secrets it doesn't need. See prompt injection in coding agents.

What policy should a team adopt for AI code review?

A short written policy prevents the two common failures: people rubber-stamping AI comments, and people ignoring the reviewer because it's noisy. Five rules cover most teams.

  1. AI review is a first pass, never the approver. A human with context approves every pull request that reaches production. Keep AI approvals off for protected branches, even where a tool offers them.
  2. The author responds to every AI comment. Fix it, or reply with one line explaining why it doesn't apply. That reply is also training data for your instruction file.
  3. High-severity findings block until resolved by a human, whether they turn out to be real or not.
  4. Tune monthly. Look at which comments led to changes and which were dismissed, then update REVIEW.md or your Copilot instructions.
  5. AI-written code gets a different reviewer than the agent that wrote it, plus a human. Agents share blind spots with themselves.

A simple metric to track is the share of AI comments that lead to a code change. If it's low, the reviewer is noisy and people will learn to ignore it; tighten severity and skip lists. Pair this with good tests, since a reviewer can't verify behavior the way a test suite does. Our guide to writing unit tests with AI covers that side.

How AI code review fits with coding agents

When coding agents write a growing share of your pull requests, review becomes the bottleneck, and AI review is how teams keep up without lowering the bar. A workflow that holds up:

  1. The agent works on a branch and runs the tests.
  2. The agent runs /code-review (or your CI reviewer runs) and fixes the high-confidence findings.
  3. The pull request opens with the review summary attached.
  4. A human reviews intent, design and the remaining findings, then merges.

Agents that run unattended, for example overnight on a remote server, produce a queue of pull requests for the morning. Orkio runs Claude Code on an always-on cloud box for exactly that pattern, so you review finished pull requests from your phone or the web instead of watching the agent work.

How to start with AI code review this week

  1. Pick one tool based on the table above. If you use Claude Code already, start with /code-review on your next branch.
  2. Turn it on for one active repository, not the whole organization.
  3. Commit a short instruction file: what counts as high severity, what to skip, three team-specific checks.
  4. Ask authors to reply to every AI comment for two weeks.
  5. Count comments that led to changes versus dismissed ones.
  6. Tighten the instructions, then roll out to more repositories.
  7. Keep a human approver on every production merge.

Frequently asked questions

Can AI replace human code reviewers?

No. AI reviewers find many local bugs quickly, but they don't know your product goals, can't judge whether a feature should exist and can miss problems that span systems. Use AI to catch mechanical issues early so human reviewers can spend their time on design, intent and risk.

Is AI code review accurate?

It varies by tool, model, review depth and how well you configure it. Expect some false positives and some missed bugs. Vendors themselves state that a review with no findings does not mean the code is correct or secure, so keep tests, static analysis and human review in place.

Does AI code review work on private repositories?

Yes. Copilot code review and Claude Code Review both work on private GitHub repositories once your organization enables them. Check each vendor's data handling terms and your company's policy before sending proprietary code to a third-party service.

How much does AI code review cost?

It depends on the tool and review depth. GitHub bills Copilot code review in AI credits, with lighter reviews costing less than balanced ones, and Claude Code Review bills per review through usage credits with Quick, Standard and Deep levels. Running a reviewer in your own CI costs whatever API tokens it uses.

Should AI-generated pull requests be reviewed by AI?

They can be, but a different reviewer setup or a human should always sign off. An agent reviewing its own work tends to share its blind spots, so pair AI review with tests and a human approver for anything that reaches production.

SOURCES

  1. GitHub Docs: About GitHub Copilot code review
  2. Claude Code docs: Code Review
  3. Claude Code docs: Customize Code Review
  4. Claude Code docs: Review locally with /code-review
  5. Claude Code docs: GitHub Actions
  • #code review
  • #github
  • #pull requests
  • #claude code
  • #copilot
Orkio team

The team building Orkio. We write about putting AI to work: coding agents that run overnight, always-on boxes and pull requests ready by morning.

Keep reading

All posts

07:06 · YOU MERGED #128

Go to sleep. Wake up to pull requests.

Start for $19 a month
© 2026 Orkio · Hosted in Europe TürkçeStatus