pr-reviewer

Automated pull request review service for self-hosted code forges. Receives webhook events, sends diffs to an LLM for review, and posts the review as a comment on the PR.

The pipeline runs against the forge.Client abstraction in internal/forge. Two backends ship today: forgejo (also Gitea, which shares its API) and gitlab. Pick one with FORGE_TYPE.

How it works

1. A repository webhook fires on PR open/synchronize/reopen/review_requested
2. pr-reviewer determines the review tier:
   • Quick scan (always): focused on bugs, security issues, and breaking changes
   • Full review (when pr-reviewer-bot is assigned as reviewer): thorough analysis including full file contents for context
3. A "working on it" comment is posted immediately with a timing estimate
4. The diff (and file contents for full reviews) is sent to the configured LLM backend
5. The pending comment is updated in-place with the review

After a full review is completed, the bot leaves the PR alone unless re-triggered by a new review_requested event or a comment mentioning @pr-reviewer-bot.

Configuration

All configuration is via environment variables:

Variable	Description	Default
FORGE_TYPE	Which forge backend to use: forgejo or gitlab	forgejo
FORGE_URL	Forge base URL. Falls back to FORGEJO_URL	http://forgejo.forgejo.svc.cluster.local
FORGE_TOKEN	API token for the bot user. Falls back to FORGEJO_TOKEN	(required)
FORGE_INSTANCE	Minted identifier for this forge. Required, permanent, never derived — see below	(required)
FORGE_DISPLAY_NAME	Human-readable label for logs and the dashboard. Safe to change	(instance id)
WEBHOOK_SECRET	Webhook verification secret. Forgejo signs the body with it; GitLab echoes it in X-Gitlab-Token	(required)
LLM_BACKEND	anthropic, vllm, or gllm	anthropic
ANTHROPIC_API_KEY	Anthropic API key	(required if backend=anthropic)
VLLM_BASE_URL	vLLM endpoint	http://vllm.vllm.svc.cluster.local:8000/v1
GLLM_BASE_URL	gllm endpoint (requires gllm ≥ v0.22.0)	http://gllm.gllm.svc.cluster.local:8000/v1
ALLOWED_REPOS	Comma-separated repo allowlist (empty = allow all)	(empty)
BOT_USERNAME	Bot's username on the forge	pr-reviewer-bot
DEFAULT_SETTLE_SECONDS	Initial settle delay before committing to quick scan	5.0
MAX_SETTLE_SECONDS	Ceiling for adaptive settle time	60.0
MAX_SETTLE_GAP_SECONDS	Gaps beyond this aren't considered near-misses	300.0
QUICK_SCAN_MAX_TOKENS	Max LLM output tokens for quick scans	2048
FULL_REVIEW_MAX_TOKENS	Max LLM output tokens for full reviews	8192
DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD	PostgreSQL connection	(from CNPG secret)

Forge instance identity

FORGE_INSTANCE is minted once per forge and then never changes. It lands in every review_events row and, together with the repo path and PR number, is what identifies a review.

It must not be derived from anything — not the forge kind, not its URL, not a position in a list. Anything derivable can be claimed by a different forge later, and that forge would silently inherit this one's review history: in-progress state, superseded reviews, and the prior review fed to the model as followup context.

Start the webhook service without it and it will refuse to run, printing a freshly minted candidate to use. The suggestion is random rather than derived, which you can confirm by running it again and getting a different one.

Use FORGE_DISPLAY_NAME for anything a human should read. That one carries all the meaning and is free to change whenever a forge is renamed or moved.

Upgrading an existing deployment: migration 000016 adds the column with an empty default, and unlabelled rows are deliberately not claimed by any instance. Backfill them explicitly once, after setting FORGE_INSTANCE:

UPDATE review_events SET forge_instance = '<your minted id>'
 WHERE forge_instance = '';

Endpoints

Webhook service (cmd/webhook)

Path	Description
POST /webhook	Forge webhook receiver
GET /health	Health check

Dashboard (cmd/dashboard)

Path	Description
GET /	Landing page (login or summary, depending on auth state)
GET /reviews	List of reviews visible to the current user
GET /reviews/{id}	Per-review detail page
GET /admin/reviews	All reviews across the system (admin only)
GET /admin/comparisons/{id}	Side-by-side A/B comparison view (admin only)
GET /admin/configs	Manage model configurations (admin only)
GET /health	Health check

Setup

Forgejo bot user

Create a user (e.g. pr-reviewer-bot) with an API token that has:

• read:repository
• write:repository
• write:issue

The bot must be added as a collaborator (write access) on each repo it reviews.

Per-repo webhook

In each repo's Settings > Webhooks, add:

• URL: http://pr-reviewer.pr-reviewer.svc.cluster.local:8080/webhook
• Content type: application/json
• Secret: must match WEBHOOK_SECRET
• Trigger: Pull Request events

Deployment

Deployed to Kubernetes via Flux from the infra repo (apps/pr-reviewer/). Two deployments share the same image:

• pr-reviewer — webhook service (ClusterIP on 8080)
• pr-reviewer-dashboard — read-only dashboard (MetalLB, serves at /)

Container images are built locally and pushed to Harbor at harbor.brooktrails.org/brooktrails/pr-reviewer.

Dashboard

An internal-only MetalLB service exposes the dashboard on the local network. Check kubectl -n pr-reviewer get svc pr-reviewer-dashboard for the assigned IP, then visit http://<ip>/.

Contributing

Changes to this repo go through branches and pull requests, not direct pushes to main. Use the following branch prefixes:

• feat/ — new features
• fix/ — bug fixes
• refactor/ — restructuring without behavior changes
• doc/ — documentation updates

The pr-reviewer bot reviews its own PRs.