TL;DR: We run two AI reviewers (Gemini Flash + Claude) on every PR via GitHub Actions. They catch real bugs, enforce conventions, and cost us nothing. Here's how we set it up and what we learned after hundreds of PRs.
The Problem With Human-Only Review
Our team is small. Most PRs sit in the review queue for hours because the one person who knows that part of the codebase is in a meeting, asleep, or working on their own feature. When the review finally happens, it's often a quick "LGTM" because the reviewer is context-switching.
We used CodeRabbit for a while and it's a solid product, but the per-seat pricing gets steep for a growing team. We also wanted more control over what gets flagged and how.
So we built our own pipeline: two AI reviewers running in parallel on every PR, each with different strengths.
The Setup
flowchart TB
pr["PR opened / updated"] --> ga["GitHub Actions"]
subgraph ga["GitHub Actions"]
gemini["Gemini Flash\n(fast, cheap, obvious issues)"]
claude["Claude Code\n(deep analysis, architecture)"]
comment["PR Comment\nwith findings"]
gemini --> comment
claude --> comment
end
Both reviewers post their findings as PR comments. The human reviewer reads the AI feedback before doing their own review. In many cases, the AI review is sufficient for low-risk changes.
Why Two Reviewers?
Different models catch different things:
| Gemini Flash | Claude | |
|---|---|---|
| Speed | ~30 seconds | ~1-2 minutes |
| Strength | Pattern matching, consistency, naming, typos | Logic errors, architectural concerns, edge cases |
| Style | Concise findings with severity labels | Detailed analysis with fix suggestions and code links |
| Cost | Free tier covers most usage | Free via GitHub App |
| False positive rate | Medium, sometimes flags pre-existing issues | Low, better at scoping to the changeset |
They frequently agree on real issues (strong signal) and occasionally catch things the other missed. When they disagree, it's usually worth investigating.
What It Actually Catches
Here are real examples from our PRs:
Bug: Wrong env var in config (caught by both)
// admin-config.service.ts
resend: {
apiKey: this.get<string>('RESEND_API_KEY', ''),
fromEmail: this.get<string>('AWS_SES_FROM_EMAIL', '[email protected]'),
// ^^^^^^^^^^^^^^^^^ wrong — should be RESEND_FROM_EMAIL
}
Both flagged that the admin service was reading the SES from-email for the Resend provider, while consumer and producer correctly read RESEND_FROM_EMAIL first. This would have caused the admin service to send from a different email address than the other services.
Logic: Silent data loss (caught by Claude)
// msg91.service.ts
export interface Msg91SendParams {
text?: string; // declared but never sent to the API
}
Claude identified that the text field was accepted by the interface but silently dropped. The MSG91 API body never included it. Emails with plain-text fallback would lose that content with no error or log.
Consistency: Default mismatch (caught by Gemini)
# .env.example says:
COMM_EMAIL_PROVIDER_ORDER=msg91,resend,ses
# But code defaults to:
const DEFAULT_EMAIL_ORDER = ['resend'];
Gemini flagged that the documented default in env examples didn't match the code-level fallback, which could confuse developers who don't set the env var.
Security: Hardcoded IPs in docs (caught by both)
| Dev backend | `https://10.0.1.130` |
| Redis | `<redis-private-ip>:6379` |
Both flagged VPC private IPs committed to the repo. Not a critical security issue (VPN-only access), but bad practice.
The GitHub Actions Workflow
Gemini Review
name: Gemini Code Review
on:
pull_request:
types: [opened, synchronize, ready_for_review]
jobs:
review:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Get diff
run: |
git diff origin/${{ github.base_ref }}...HEAD > diff.txt
- name: Review with Gemini
env:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
run: |
# Send diff to Gemini API with review prompt
# Parse response and post as PR comment
node scripts/gemini-review.mjs
- name: Post comment
uses: actions/github-script@v7
with:
script: |
const review = require('./review-output.json');
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: review.markdown
});
Claude Review
Claude runs via the official GitHub App. No custom workflow needed:
- Install the Claude GitHub App
- Configure it to run on PRs
- It automatically reviews every PR and posts findings as a comment
No API keys, no scripts, no tokens.
The Review Prompt
For Gemini, the prompt matters. Here's what we use (simplified):
You are reviewing a pull request for a NestJS monorepo.
Context:
- Framework: NestJS 11, Fastify, Drizzle ORM, PostgreSQL
- Three services: consumer, producer, admin sharing a common library
- Conventional commits enforced
Review the following diff and report findings in this format:
### Findings
RED HIGH — file:line (category)
> Description and suggestion
YELLOW MEDIUM — file:line (category)
> Description and suggestion
GREEN LOW — file:line (category)
> Description and suggestion
### What's Well Done
- Positive observations
Categories: bug, security, performance, maintainability, logic
Rules:
- Only flag issues in changed lines, not pre-existing code
- Don't flag style issues that a linter would catch
- Don't suggest adding comments or documentation
- Be specific — include file paths and line numbers
- If the code is good, say so and approve
Telling it what NOT to flag matters more than what to flag. Without constraints, AI reviewers flood PRs with style nits and documentation suggestions.
Project-Level Review Guidelines
The prompt gets you 70% of the way. The other 30% is project context that the AI reviewer can't infer from a diff alone.
We keep a CLAUDE.md file in the repo root. Claude picks this up automatically. For Gemini, we append the file contents to the review prompt. The file covers:
- Architecture decisions: "We use Drizzle ORM, not TypeORM. Don't suggest TypeORM patterns."
- Conventions: "All config values go through the ConfigService. Direct
process.envaccess is not allowed outside config files." - Known patterns: "The producer service is single-replica by design. Don't flag it as a scaling concern."
- Error handling: "We use NestJS exception filters. Don't suggest try/catch in controllers."
- Testing: "Integration tests hit a real database. Don't suggest mocking the database layer."
Example from our CLAUDE.md:
## Review Guidelines
- Config: all env vars accessed via ConfigService, never process.env directly
- Validation: DTOs use class-validator decorators, no manual validation in handlers
- Queues: Redis-backed via BullMQ, producer pushes jobs, workers process them
- Auth: JWT with Passport, guards on controllers not on services
- DB: Drizzle ORM with PostgreSQL, migrations in /drizzle folder
- Don't flag: single-replica producer (by design), shared common library imports
This reduces false positives significantly. Before adding the file, Gemini would flag our single-replica producer on almost every PR that touched it. After adding the "by design" note, those findings stopped.
The file also helps new team members. It's a living document of "how we do things here" that both humans and AI reviewers reference. When we change a convention, we update the file and the AI reviewer adapts immediately.
The Audit Step
AI reviewers have false positives. We don't blindly apply every suggestion. For each finding, we run through:
Issue N: [SEVERITY] description
├── AI says: <their claim>
├── Actual code: <what the code actually does>
├── Verdict: FIX | SKIP | PARTIAL
└── Reasoning: <why>
Common skip reasons:
- Pre-existing issue: not introduced by this PR
- By design: the AI doesn't understand the business context
- Conflicts with project conventions: our CLAUDE.md says otherwise
- False positive: the AI misread the code flow
In our experience, about 60-70% of findings are actionable. The rest are skipped with a reason logged.
Cost
| Tool | Monthly Cost | Notes |
|---|---|---|
| CodeRabbit | ~$12/seat/month | Per-seat pricing adds up |
| Gemini Flash | $0 | Free tier covers ~1500 reviews/month |
| Claude GitHub App | $0 | Free for open source and small teams |
| Our setup | $0 | Both reviewers are free tier |
Even if you outgrow the free tiers, the API costs are negligible. A typical PR diff is 2-5K tokens. At Gemini Flash pricing that's fractions of a cent per review.
What We'd Do Differently
Start with one reviewer, not two. We added Claude second after a month of Gemini-only. Two reviewers generate a lot of comments. Make sure your team actually reads them before doubling the volume.
Tune the prompt aggressively. Our first prompt produced 15+ findings per PR, most of them noise. We iterated for two weeks until the signal-to-noise ratio was acceptable. The key additions were "only flag changed lines" and "don't suggest documentation."
Don't block merge on AI review. AI reviews are advisory. We never gate merges on AI approval. They're a second pair of eyes, not a gatekeeper. Human judgment remains the final call.
Keep a skip log. When you skip an AI finding, note why. After a month, review the skip log. If you're consistently skipping the same category, update the prompt to stop flagging it.
The Workflow in Practice
- Developer opens PR
- GitHub Actions triggers both reviewers (~1-2 minutes)
- AI findings appear as PR comments
- Developer reads findings, fixes real issues, skips false positives
- Human reviewer sees the AI feedback + the developer's responses
- Human review is faster because the obvious issues are already surfaced
The human reviewer's job shifts from "find bugs" to "validate architecture and business logic."
Should You Build This?
Yes, if:
- Your team is small and reviews are a bottleneck
- You want consistent review quality regardless of who's reviewing
- You want to reduce tooling costs
- You want to customize what gets flagged
No, if:
- You need inline suggestions with one-click apply (CodeRabbit does this better)
- You want review-as-a-service with zero setup
- Your team doesn't read PR comments anyway
The setup is an afternoon. Most of the ongoing work is tuning the prompt until the findings are worth reading.