brooktrails/pr-reviewer
2
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

feat: surface A/B configs and verdicts on /admin/reviews #55

Merged
rcsheets merged 1 commit from feat/admin-reviews-ab-display into main 2026-05-01 12:13:35 +00:00
Conversation 1 Commits 1 Files changed 4 +100 -11
rcsheets commented 2026-05-01 12:11:54 +00:00
Owner
Copy link

The all-reviews list previously showed only the primary config's
verdict and model, hiding the existence of any shadow configs that
ran alongside. Admins running A/B comparisons (Sonnet primary + Opus
shadow, etc.) had to drill into each review to see whether the two
agreed.

Replace the single Verdict + Model columns with a Configs column
that lists every config that ran on the PR — primary first, then
shadows in timestamp order — each as a verdict-colored pill carrying
the display name and verdict label. Disagreement is now visible
inline: when configs return different verdicts, the pills' verdict
colors mismatch and the row stands out at a glance.

Adds tracker.ConfigOutcomesForGroups to bulk-fetch the
comparison-group siblings for the listed primaries in a single query
(no N+1). Falls back to a single-pill rendering using the stored
backend + verdict when a row has no comparison group, so legacy or
shadow-less reviews still render sensibly.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

The all-reviews list previously showed only the primary config's verdict and model, hiding the existence of any shadow configs that ran alongside. Admins running A/B comparisons (Sonnet primary + Opus shadow, etc.) had to drill into each review to see whether the two agreed. Replace the single Verdict + Model columns with a Configs column that lists every config that ran on the PR — primary first, then shadows in timestamp order — each as a verdict-colored pill carrying the display name and verdict label. Disagreement is now visible inline: when configs return different verdicts, the pills' verdict colors mismatch and the row stands out at a glance. Adds tracker.ConfigOutcomesForGroups to bulk-fetch the comparison-group siblings for the listed primaries in a single query (no N+1). Falls back to a single-pill rendering using the stored backend + verdict when a row has no comparison group, so legacy or shadow-less reviews still render sensibly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: surface A/B configs and verdicts on /admin/reviews
All checks were successful
ci / check (pull_request) Successful in 44s
Details
2dc96b09de 
The all-reviews list previously showed only the primary config's
verdict and model, hiding the existence of any shadow configs that
ran alongside. Admins running A/B comparisons (Sonnet primary + Opus
shadow, etc.) had to drill into each review to see whether the two
agreed.

Replace the single Verdict + Model columns with a Configs column
that lists every config that ran on the PR — primary first, then
shadows in timestamp order — each as a verdict-colored pill carrying
the display name and verdict label. Disagreement is now visible
inline: when configs return different verdicts, the pills' verdict
colors mismatch and the row stands out at a glance.

Adds tracker.ConfigOutcomesForGroups to bulk-fetch the
comparison-group siblings for the listed primaries in a single query
(no N+1). Falls back to a single-pill rendering using the stored
backend + verdict when a row has no comparison group, so legacy or
shadow-less reviews still render sensibly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pr-reviewer-bot commented 2026-05-01 12:11:55 +00:00
Collaborator
Copy link

Automated review by pr-reviewer v0.23.0 | Safety Check | Claude Sonnet 4 | tracking id r-f4988b-270d08
This is an AI-generated review and may contain mistakes.

Status: ✅ Completed

✅ Verdict: LGTM — Clean feature addition that improves admin visibility into A/B testing configurations without introducing security risks or breaking changes.
Complexity: moderate

Safety Check Review

This PR adds A/B configuration display functionality to the admin reviews page. The changes appear safe:

Key Changes:

  • Adds bulk fetching of config outcomes to avoid N+1 queries
  • Updates UI to show multiple configs per review with verdict-colored pills
  • Graceful degradation when comparison groups aren't available

Safety Assessment:

  • ✅ No security vulnerabilities detected
  • ✅ No hardcoded secrets or credentials
  • ✅ Database queries use parameterized statements
  • ✅ Error handling preserves functionality (falls back gracefully)
  • ✅ Admin-gated functionality remains properly protected
  • ✅ No breaking changes to existing functionality

The code follows existing patterns, handles errors appropriately, and maintains backward compatibility with legacy reviews that lack comparison groups.

*Automated review by [pr-reviewer](https://git.brooktrails.org/brooktrails/pr-reviewer) v0.23.0 | Safety Check | Claude Sonnet 4 | tracking id `r-f4988b-270d08`* *This is an AI-generated review and may contain mistakes.* **Status:** ✅ Completed --- **✅ Verdict: LGTM** — Clean feature addition that improves admin visibility into A/B testing configurations without introducing security risks or breaking changes. **Complexity:** moderate # Safety Check Review This PR adds A/B configuration display functionality to the admin reviews page. The changes appear safe: **Key Changes:** - Adds bulk fetching of config outcomes to avoid N+1 queries - Updates UI to show multiple configs per review with verdict-colored pills - Graceful degradation when comparison groups aren't available **Safety Assessment:** - ✅ No security vulnerabilities detected - ✅ No hardcoded secrets or credentials - ✅ Database queries use parameterized statements - ✅ Error handling preserves functionality (falls back gracefully) - ✅ Admin-gated functionality remains properly protected - ✅ No breaking changes to existing functionality The code follows existing patterns, handles errors appropriately, and maintains backward compatibility with legacy reviews that lack comparison groups.
rcsheets deleted branch feat/admin-reviews-ab-display 2026-05-01 12:13:36 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
brooktrails/pr-reviewer!55
No description provided.