Building a code review agent with Claude and GitHub Actions

By Bianca Moreira · 17 July 2026103 views
Building a code review agent with Claude and GitHub Actions
#claude#github-actions#code-review#ai-agent#devops

What automated code review should and should not do

Automated code review with LLMs sits in an awkward middle ground. It's not a linter — it shouldn't replace ESLint or type checking. It's not a human reviewer — it can't understand your team's product context or evaluate business logic. What it can do reliably is surface patterns that are easy to miss in manual review: security issues in authentication code, N+1 query patterns, missing error handling, and inconsistency with the surrounding codebase style.

The agent we'll build in this article does exactly that. It runs on every pull request, reads the diff, analyses each changed file in context, and posts inline comments directly on the PR. It is scoped tightly: security vulnerabilities, performance anti-patterns, missing error handling, and API contract violations. It stays out of style preferences and business logic.

By the end you'll have a working GitHub Actions workflow, a TypeScript agent that calls the Claude API, and a comment-posting integration with the GitHub REST API.

Architecture overview

The flow is:

  1. A PR is opened or pushed to
  2. GitHub Actions triggers the review workflow
  3. The workflow fetches the PR diff via the GitHub API
  4. For each changed file, it sends the original and modified version to Claude with a structured review prompt
  5. Claude returns a structured JSON response with findings, each containing a file path, line number, severity, and description
  6. The workflow posts inline review comments to the PR using the GitHub Review API

Keeping the agent's output structured (JSON) is important — it makes comment-posting deterministic and prevents the agent from writing prose that is hard to parse into file/line coordinates.

Setting up the GitHub Actions workflow

Create .github/workflows/ai-code-review.yml:

name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - name: Install dependencies
        run: npm ci
        working-directory: .github/ai-review

      - name: Run AI review
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          REPO_OWNER: ${{ github.repository_owner }}
          REPO_NAME: ${{ github.event.repository.name }}
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: node dist/review.js
        working-directory: .github/ai-review

The pull-requests: write permission is required to post review comments. contents: read is needed to read the diff. Keep ANTHROPIC_API_KEY in GitHub repository secrets.

The review agent: fetching the diff

Create .github/ai-review/src/review.ts. Start with the diff fetcher:

import { Octokit } from "@octokit/rest";

const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });

interface FileDiff {
  filename: string;
  status: "added" | "modified" | "deleted" | "renamed";
  patch: string; // unified diff format
  additions: number;
  deletions: number;
}

async function getPRDiff(): Promise<FileDiff[]> {
  const owner = process.env.REPO_OWNER!;
  const repo = process.env.REPO_NAME!;
  const pull_number = parseInt(process.env.PR_NUMBER!, 10);

  const { data: files } = await octokit.pulls.listFiles({
    owner,
    repo,
    pull_number,
    per_page: 100,
  });

  return files
    .filter((f) => f.status !== "deleted" && f.patch)
    .filter((f) => shouldReviewFile(f.filename))
    .map((f) => ({
      filename: f.filename,
      status: f.status as FileDiff["status"],
      patch: f.patch ?? "",
      additions: f.additions,
      deletions: f.deletions,
    }));
}

function shouldReviewFile(filename: string): boolean {
  const reviewable = [".ts", ".tsx", ".js", ".jsx", ".py", ".go", ".java", ".rb"];
  const skip = [
    "package-lock.json",
    "yarn.lock",
    "pnpm-lock.yaml",
    ".min.js",
    ".generated.",
    "__snapshots__",
  ];

  if (skip.some((pattern) => filename.includes(pattern))) return false;
  return reviewable.some((ext) => filename.endsWith(ext));
}

Filter out lock files, generated code, and test snapshots — reviewing these wastes tokens and produces noise.

The review agent: analysing with Claude

Define the expected output structure and call Claude with a prompt that constrains it to return JSON:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

interface ReviewFinding {
  filename: string;
  line: number; // line in the new file (right side of diff)
  severity: "critical" | "major" | "minor" | "info";
  category: "security" | "performance" | "error-handling" | "api-contract" | "correctness";
  title: string;
  description: string;
  suggestion: string;
}

interface ReviewResult {
  findings: ReviewFinding[];
  summary: string;
}

const REVIEW_SYSTEM_PROMPT = `You are a senior software engineer performing a focused code review.

Your job is to identify real problems only. Do NOT comment on:
- Code style or formatting (that is handled by linters)
- Minor naming preferences
- Business logic you do not have context for

DO comment on:
- Security vulnerabilities: SQL injection, XSS, hardcoded secrets, missing auth checks, insecure crypto
- Performance issues: N+1 queries, missing indexes, synchronous blocking in async paths, unbounded loops
- Error handling gaps: unhandled promise rejections, missing try/catch around I/O, swallowed exceptions
- API contract violations: calling external APIs without timeout, missing input validation, incorrect HTTP status codes
- Clear correctness bugs: off-by-one errors, wrong operator precedence, undefined variable access

Return ONLY valid JSON matching this schema, with no prose before or after:
{
  "findings": [
    {
      "filename": "src/auth/login.ts",
      "line": 42,
      "severity": "critical",
      "category": "security",
      "title": "SQL injection via string interpolation",
      "description": "User input is directly interpolated into the SQL query without parameterisation.",
      "suggestion": "Use a parameterised query: db.query('SELECT * FROM users WHERE email = $1', [email])"
    }
  ],
  "summary": "2 critical issues found in auth module. SQL injection in login.ts and missing CSRF protection in session.ts."
}

If there are no issues, return { "findings": [], "summary": "No issues found." }`;

async function reviewFile(file: FileDiff): Promise<ReviewResult> {
  const prompt = `Review this pull request change.

File: ${file.filename}
Status: ${file.status}

Diff (unified format):
\`\`\`diff
${file.patch}
\`\`\`

Identify any issues in the ADDED lines (lines starting with +). For each issue, provide the line number in the new file where the problem occurs.`;

  const response = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 2048,
    system: REVIEW_SYSTEM_PROMPT,
    messages: [{ role: "user", content: prompt }],
  });

  const textBlock = response.content.find((b) => b.type === "text");
  const text = textBlock?.type === "text" ? textBlock.text : "{}";

  try {
    return JSON.parse(text) as ReviewResult;
  } catch {
    console.error("Failed to parse Claude response as JSON:", text);
    return { findings: [], summary: "Review parsing failed." };
  }
}

Using claude-sonnet-4-5 rather than Opus keeps costs manageable for high-frequency PR workflows. Sonnet performs well on code analysis tasks. Switch to Opus for repositories where security review quality is paramount.

Converting diff line numbers to GitHub comment positions

GitHub's review API uses "position" — the line number within the unified diff — not the line number in the file. This is a common source of confusion. Convert between them:

function buildLineToPositionMap(patch: string): Map<number, number> {
  const lines = patch.split("\n");
  const map = new Map<number, number>();
  let currentLine = 0;
  let position = 0;

  for (const line of lines) {
    position++;

    if (line.startsWith("@@")) {
      // Parse the hunk header: @@ -old_start,old_count +new_start,new_count @@
      const match = line.match(/@@ -\d+(?:,\d+)? \+(\d+)/);
      if (match) {
        currentLine = parseInt(match[1], 10) - 1;
      }
    } else if (line.startsWith("+")) {
      currentLine++;
      map.set(currentLine, position);
    } else if (!line.startsWith("-")) {
      currentLine++;
    }
  }

  return map;
}

This maps each new-file line number to the position value required by the GitHub API.

Posting review comments to the PR

interface PendingComment {
  path: string;
  position: number;
  body: string;
}

function severityEmoji(severity: ReviewFinding["severity"]): string {
  return { critical: "🔴", major: "🟠", minor: "🟡", info: "🔵" }[severity];
}

function buildCommentBody(finding: ReviewFinding): string {
  return [
    `${severityEmoji(finding.severity)} **${finding.title}** (${finding.severity})`,
    "",
    finding.description,
    "",
    "**Suggestion:**",
    finding.suggestion,
    "",
    `_Category: ${finding.category}_`,
  ].join("\n");
}

async function postReview(
  allFindings: ReviewFinding[],
  summary: string,
  fileDiffs: FileDiff[]
): Promise<void> {
  const owner = process.env.REPO_OWNER!;
  const repo = process.env.REPO_NAME!;
  const pull_number = parseInt(process.env.PR_NUMBER!, 10);
  const commitId = process.env.HEAD_SHA!;

  const comments: PendingComment[] = [];

  for (const finding of allFindings) {
    const fileDiff = fileDiffs.find((f) => f.filename === finding.filename);
    if (!fileDiff) continue;

    const lineMap = buildLineToPositionMap(fileDiff.patch);
    const position = lineMap.get(finding.line);
    if (!position) {
      console.warn(
        `Could not find position for ${finding.filename}:${finding.line}`
      );
      continue;
    }

    comments.push({
      path: finding.filename,
      position,
      body: buildCommentBody(finding),
    });
  }

  const criticalCount = allFindings.filter((f) => f.severity === "critical").length;
  const reviewEvent: "APPROVE" | "REQUEST_CHANGES" | "COMMENT" =
    criticalCount > 0 ? "REQUEST_CHANGES" : "COMMENT";

  const reviewBody = [
    "## AI Code Review",
    "",
    summary,
    "",
    `**${allFindings.length} finding(s)** — ${criticalCount} critical, ` +
      `${allFindings.filter((f) => f.severity === "major").length} major, ` +
      `${allFindings.filter((f) => f.severity === "minor").length} minor`,
    "",
    "_This review was generated automatically. Human review is still required._",
  ].join("\n");

  await octokit.pulls.createReview({
    owner,
    repo,
    pull_number,
    commit_id: commitId,
    body: reviewBody,
    event: reviewEvent,
    comments,
  });
}

The review uses REQUEST_CHANGES if any critical findings are present, and COMMENT otherwise. This prevents merging PRs with critical security issues while not blocking PRs that only have minor suggestions.

The main orchestration function

async function main() {
  console.log("Fetching PR diff...");
  const files = await getPRDiff();
  console.log(`Found ${files.length} reviewable files`);

  if (files.length === 0) {
    console.log("No reviewable files in this PR. Skipping.");
    return;
  }

  const allFindings: ReviewFinding[] = [];
  const summaries: string[] = [];

  for (const file of files) {
    if (file.patch.length < 20) continue; // Skip trivially small changes

    console.log(`Reviewing ${file.filename}...`);
    try {
      const result = await reviewFile(file);
      allFindings.push(...result.findings);
      if (result.summary && result.summary !== "No issues found.") {
        summaries.push(`**${file.filename}**: ${result.summary}`);
      }
    } catch (err) {
      console.error(`Failed to review ${file.filename}:`, err);
    }

    // Rate limit: 10 requests per minute for Claude API
    await new Promise((r) => setTimeout(r, 6000));
  }

  const overallSummary =
    summaries.length > 0 ? summaries.join("\n") : "No significant issues found.";

  await postReview(allFindings, overallSummary, files);
  console.log(`Review complete. Posted ${allFindings.length} finding(s).`);
}

main().catch((err) => {
  console.error("Review agent failed:", err);
  process.exit(1);
});

The 6-second delay between file reviews is a conservative rate limit. Adjust based on your Claude API tier's rate limits. If you have a high-throughput API tier, you can parallelise file reviews to cut total review time.

Caching the system prompt for large PRs

For PRs with many files, add prompt caching to avoid paying full input token cost for the system prompt on each file review:

async function reviewFileWithCache(file: FileDiff): Promise<ReviewResult> {
  const response = await client.messages.create({
    model: "claude-sonnet-4-5",
    max_tokens: 2048,
    system: [
      {
        type: "text",
        text: REVIEW_SYSTEM_PROMPT,
        cache_control: { type: "ephemeral" },
      },
    ],
    messages: [
      {
        role: "user",
        content: `Review this change in ${file.filename}:\n\n\`\`\`diff\n${file.patch}\n\`\`\``,
      },
    ],
  });

  const textBlock = response.content.find((b) => b.type === "text");
  const text = textBlock?.type === "text" ? textBlock.text : "{}";
  return JSON.parse(text) as ReviewResult;
}

With the 600-token system prompt cached, a 20-file PR pays full input cost once and cache read cost (10%) for the remaining 19 files — roughly 80% reduction in input token costs for that PR.

Preventing review spam

One common complaint with automated reviewers is that they re-comment on the same issues across multiple pushes. Track reviewed commits and skip files that haven't changed since the last review:

async function getAlreadyReviewedFiles(): Promise<Set<string>> {
  const owner = process.env.REPO_OWNER!;
  const repo = process.env.REPO_NAME!;
  const pull_number = parseInt(process.env.PR_NUMBER!, 10);

  const { data: reviews } = await octokit.pulls.listReviews({
    owner,
    repo,
    pull_number,
  });

  const botReviews = reviews.filter(
    (r) =>
      r.user?.type === "Bot" &&
      r.body?.includes("AI Code Review")
  );

  if (botReviews.length === 0) return new Set();

  // Get all comments from previous bot reviews
  const { data: comments } = await octokit.pulls.listReviewComments({
    owner,
    repo,
    pull_number,
  });

  const reviewedFiles = new Set(
    comments
      .filter((c) => botReviews.some((r) => r.id === c.pull_request_review_id))
      .map((c) => c.path)
  );

  return reviewedFiles;
}

Use this set to skip files that were already reviewed on a previous push, unless those files have new changes in the current push (check by comparing file SHAs between runs).

Configuring per-repository review scope

Add a .ai-review.json config file to repositories to customise review behaviour without modifying the agent:

{
  "categories": ["security", "error-handling"],
  "minSeverity": "major",
  "skipPaths": ["src/migrations/", "src/__generated__/"],
  "extraContext": "This is a financial services application. Pay extra attention to money calculations and rounding."
}

Load this config at the start of the review and append extraContext to the system prompt (before the cache boundary) and filter findings by minSeverity. This lets each team tune the review focus without needing separate workflow files.

The result is an automated reviewer that surfaces real issues — not style nitpicks — comments precisely on the relevant lines, and integrates cleanly into the PR review process without replacing human judgment.

Comments

No comments yet. Be the first!

Sign in to leave a comment.