AI-Powered Code Reviews with OpenRouter – Complete PR-Agent Setup Guide



This content originally appeared on DEV Community and was authored by TOMOAKI ishihara

👋 Introduction

Code reviews take time, and large teams often struggle with PR . PR-Agent (now Qodo Merge) uses AI to automatically review pull requests, generate descriptions, and suggest improvements.

This guide shows you how to set up PR-Agent with OpenRouter, giving you cost-effective access to multiple AI models (GPT-4o, Claude, Gemini, etc.) through a unified API.

📌 Why OpenRouter?

Challenges with Direct OpenAI API

  • Single provider dependency (OpenAI models only)
  • Complex management of multiple provider APIs
  • Difficult to compare pricing across different model

OpenRouter Benefits

  • Unified Multi-Model API – Access GPT-4o, Claude, Gemini, Llama, etc. through one interface
  • Flexible Model Selection – Choose the right model for each task, optimizing cost-effectiveness
  • Early Access to Latest Models – Quick access to new models like GPT-5
  • Transparent Pricing – Clear pricing for each model (5.5% fee on credit purchases)

Prerequisites

  • GitHub repository (private or public)
  • OpenRouter account (free to create)
  • GitHub Actions permissions

Step 1: OpenRouter Setup

1.1 Account Creation and API Key

  1. Visit OpenRouter and create an account
  2. Purchase Credits (Important)
    • Go to the “Credits” page in your dashboard
    • Minimum purchase is $5 (5.5% fee applies to credit purchases)
    • Auto-recharge option available for convenience
    • ⚠ Free Tier: New accounts receive a small free credit allowance, but additional purchases are needed for production use(FAQ: What free tier options exist?)
  3. In the dashboard, go to “API Keys” and generate a new key
  4. Copy and securely store the key

1.2 Available Models

OpenRouter provides access to models like:

openai/gpt-5           # Latest model (2025 release)
openai/gpt-4o          # Balanced performance
anthropic/claude-3-opus-20240229  # High-quality analysis
google/gemini-pro      # Google's model
meta-llama/llama-2-70b-chat  # Open source

Check pricing at OpenRouter Models.

Step 2: GitHub Actions Workflow Setup

2.1 Configure Secrets

In your GitHub repository:

  1. Go to Settings → Secrets and variables → Actions
  2. Click “New repository secret”
  3. Add the following secret:
Name: OPENROUTER_API_KEY
Value: or-xxxxxxxx (your OpenRouter API key)

2.2 Create Workflow File

Create .github/workflows/pr_agent.yml

name: Code Review(OpenRouter)

on:
  pull_request:
  pull_request_review_comment:
    types: [created]

# Concurrency control
concurrency:
  group: ${{ github.repository }}-${{ github.event.number || github.head_ref || github.sha }}-${{ github.workflow }}-${{ github.event_name == 'pull_request_review_comment' && 'pr_comment' || 'pr' }}
  cancel-in-progress: ${{ github.event_name != 'pull_request_review_comment' }}

jobs:
  pr_agent_job:
    if: |
      github.event.sender.type != 'Bot' &&
      github.event_name == 'pull_request' &&
      github.event.pull_request.changed_files < 50
    runs-on: ubuntu-latest
    permissions:
      issues: write
      pull-requests: write
      contents: write
      id-token: write
    name: Run pr agent on every pull request
    steps:
      - name: PR Agent action step with OpenRouter
        id: pragent
        uses: qodo-ai/pr-agent@main
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENROUTER.KEY: ${{ secrets.OPENROUTER_API_KEY }} # OpenRouter API認証
          github_action_config.auto_review: "true" # enable\disable auto review
          github_action_config.auto_describe: "true" # enable\disable auto describe
          github_action_config.auto_improve: "true" # enable\disable auto improve

Step 3: Configuration File

Create .pr_agent.toml in your repository root.

[config]
model = "openai/gpt-5" # I specified gpt-5, but an error occurred. The correct setting is "openai/gpt-5".
fallback_models = ["openai/gpt-4o", "anthropic/claude-opus-4.1"]
ai_timeout = 300
custom_model_max_tokens = 200000 # Critical! Errors without this
response_language = "en-US"
ignore_pr_title = ["^\\[Auto\\]", "^Auto"]
ignore_pr_labels = ['invalid']

[ignore]
glob = ['dist/**']

[github_action_config]
# set as env var in .github/workflows/pr-agent.yaml
#auto_review = true
#auto_improve = true
#auto_describe = true

[pr_reviewer]
extra_instructions = """\
(1) Act as a highly experienced software engineer
(2) Provide thorough review of code, documents, and articles
(3) Suggest concrete code snippets for improvement

(4) **Never** comment on indentation, whitespace, blank lines, or other purely stylistic issues unless they change program semantics.

(5) **Priority Review Areas - Check systematically:**
- **Security**: Plaintext passwords, SQL injection, input validation, exception leaks
- **Error Handling**: Bare except clauses, missing try-catch, silent error suppression  
- **Resource Management**: Missing context managers (with statements), unclosed connections/files
- **Type Safety**: Missing type hints, incorrect type usage, unjustified Any types
- **Performance**: Inefficient algorithms (O(n²) or worse), unnecessary loops, memory leaks
- **Code Quality**: Magic numbers, unclear variable names, unused imports/variables
- **API Design**: Missing input validation, no error responses, required field checks
- **Architecture**: Single responsibility violations, tight coupling, global state usage

(6) Focus on concrete, actionable issues with specific code examples and fix recommendations.
"""
num_code_suggestions = 5
inline_code_comments = true
ask_and_reflect = true

[pr_description]
extra_instructions = "Generate clear and comprehensive descriptions"
generate_ai_title = true
add_original_user_description = false 
publish_description_as_comment = true

[pr_code_suggestions]
extra_instructions = "Provide actionable code suggestions with examples"
commitable_code_suggestions = true
demand_code_suggestions_self_review=true

Step 4: Testing the Setup

4.1 Create Test PR

Create a PR with intentionally problematic code.

# Missing type hints
def fizz_buzz(n):
    for i in range(1, n):
        if i % 15 == 0:
            print("FizzBuzz")
        elif i % 3 == 0:
            print("Fizz") 
        elif i % 5 == 0:
            print("Buzz")
        else:
            printo(i)  # Intentional bug

# Security issue
def add_user(name, password):
    users = []
    user = {"name": name, "password": password}  # Plaintext password
    users.append(user)

4.2 Expected Output

PR-Agent will provide.

Auto-generated PR Description:

  • Summary of changes
  • File-by-file change details
  • Impact analysis

Code Review Comments:

  • Fix for printo(i)print(i)
  • Type hint suggestions
  • Security warning about plaintext passwords
  • Performance improvement suggestions

Step 5: Advanced Configuration

5.1 Cost Optimization

Use different models for different tasks.

[config]
model = "openai/gpt-4o-mini"  # For lightweight tasks
model_turbo = "openai/gpt-5"  # For complex analysis
fallback_models = ["anthropic/claude-3.7-sonnet", "google/gemini-2.5-pro"]

5.2 Team-Specific Rules

[pr_reviewer]
extra_instructions = """\
(1) Company-specific checks:
- Use structured logging for all log outputs
- External API calls must have timeout settings
- Database access must use transactions
- Never hardcode secrets (API keys, passwords, etc.)
"""

5.3 File Exclusions

[ignore]
glob = ['dist/**', '*.min.js', 'node_modules/**', '__pycache__/**']

[config]
ignore_pr_title = ["^\\[Auto\\]", "^WIP:", "^Draft:"]
ignore_pr_labels = ['work-in-progress', 'do-not-review']

Troubleshooting

Common Issues and Solutions

1. Model not recognized

Error: Model openai/gpt-5 is not defined in MAX_TOKENS
Solution: Add custom_model_max_tokens = 200000 to config

This error occurs when using a model that PR-Agent doesn’t recognize. The models supported by PR-Agent are listed in this file. If your chosen model isn’t in this list, you need to set the custom_model_max_tokens configuration.

2. Inconsistent language responses

[config]
response_language = "en-US"  # Be specific

[pr_reviewer]
extra_instructions = "Always respond in English and explain technical terms clearly"

3. GitHub Actions permission errors

permissions:
  issues: write
  pull-requests: write
  contents: write  # Add this line
  id-token: write

4. Timeouts on large PRs

[config]
ai_timeout = 600  # Extend to 10 minutes
large_patch_policy = "clip"  # Split large PRs

Cost Management Best Practices

1. Appropriate Model Selection

Task Recommended Model Pricing (approx.)
Simple reviews gpt-4o-mini $0.15/1M tokens
Detailed analysis gpt-4o $2.5/1M tokens
Highest quality openai/gpt-5 $1.25/1M tokens

2. PR Size Limits

if: github.event.pull_request.changed_files < 50  # Limit file count

3. Control Automatic Execution

[config]
ignore_pr_labels = ['draft', 'wip', 'skip-ai-review']
ignore_pr_title = ["^\\[WIP\\]", "^Draft:", "^Auto"]

Impact Measurement

Before/After Comparison Metrics

Quantitative Metrics:

  • Reduced code review time
  • Increased bug detection
  • Earlier security issue identification
  • Faster PR merge times

Qualitative Metrics:

  • Reduced reviewer burden
  • Improved code quality
  • Learning effect (developer skill improvement)

Real-World Results Example

Before: Average review time 2 hours/PR
After: Average review time 45 minutes/PR (62% reduction)

Security issue detection:
- Before: 2 issues/month average
- After: 8 issues/month average (4x increase)

📝 Conclusion

Setting up PR-Agent with OpenRouter provides these benefits.

✅ Cost Performance – Use multiple models through OpenRouter for cost optimization

✅ Quality Improvement – Consistent review standards reduce oversights

✅ Efficiency – Significant reduction in review time

✅ Learning Effect – Developers learn best practices from AI feedback

✅ Flexibility – Customizable for team-specific requirements

We recommend starting with a small project for testing and gradually expanding. The configuration file allows extensive customization to match your team’s needs.

📚 References

Hope this guide helps improve your code review process! Feel free to share questions or suggestions in the comments.


This content originally appeared on DEV Community and was authored by TOMOAKI ishihara