Automating PR Gatekeeping: How Risk Tiers Redefine Agent Code Reviews in .NET
A look at how risk-tier PR gates balance automated validation and human review to handle high volumes of AI-generated code in .NET repositories.
As autonomous coding agents generate pull requests at unprecedented volumes, traditional line-by-line manual code review is proving unsustainable for software teams. In response to this shifting workload, a practical experiment in an ASP.NET Core 8 repository demonstrates how developers can replace passive diff speed-reading with an automated risk-tiering gate. Inspired by internal practices detailed by Anthropic's Addy Osmani, Morgan Westlee Lunt, and Qing Wang—where pull request volume grew significantly—the system evaluates code changes based on diff impact and file paths rather than author identity.
Designing a Three-Tier Automated Gate
The framework uses a standalone C# console application named riskgate alongside GitHub Actions workflows to classify incoming pull requests into low, medium, or high risk tiers. Each tier dictates both mandatory evidence checks and the required number of human approvals.
- Low Tier: Applies to changes confined strictly to documentation, Markdown files, tests, or feature-flagged experimental endpoints under 300 total changed lines or 15 files. Low-risk PRs require zero human approvals, merging automatically once automated checks and template validation pass.
- Medium Tier: Triggers when changes touch general source code (
src/), configuration files (appsettings*.json), unknown directory paths, or exceed 300 lines or 15 files. It also catches PRs where test code shrinks by more than 20 lines—preventing agents from fixing failing builds by deleting tests. Medium PRs require one human approval and verifieddotnet testoutput pasted into the PR body. - High Tier: Reserved for sensitive components including authentication modules, database migrations, application startup logic (
Program.cs), package dependencies, CI configurations, and diffs exceeding 800 lines. High-tier changes require sign-off from two human reviewers, unit test output, and a recorded run (such as acurl -itranscript or visual proof).
To prevent agents from pushing unreviewed changes directly to remote branches, the gate operates as a server-side rule in GitHub Actions rather than relying solely on client-side pre-push git hooks.
Filtering Noise in Advisory Model Reviews
In addition to deterministic rules, the pipeline incorporates an optional preliminary review step using local language models through Ollama (specifically running qwen2.5-coder:14b). Drawing lessons from enterprise implementations such as Uber’s multi-agent uReview pipeline, a primary challenge in automated code review is filtering out irrelevant or hallucinated feedback.
To maintain signal quality, the tool strips out LLM findings unless they explicitly quote exact changed lines and valid file paths present within the pull request diff. This advisory pass runs purely to assist human reviewers and never blocks automated workflows or fails CI builds on its own. Furthermore, the overall review process adopts strict iteration limits—capping back-and-forth review cycles at three rounds—to ensure pull requests converge quickly instead of accumulating dozens of automated revisions.
What it means for developers
The rise of agentic software development requires teams to rethink how human attention is allocated during code reviews. By delegating routine documentation and isolated updates to automated guardrails, developers can preserve focus for high-risk architectural decisions, security boundaries, and core business logic.
Key takeaways for engineering teams adopting agentic workflows include:
- Separate ownership from approval depth: While standard mechanisms like
CODEOWNERSdictate who is eligible to review specific paths, automated gates should dictate how many approvals are required based on change magnitude and path sensitivity. - Demand structural proof over summary claims: Automated templates should enforce clear evidence—such as raw test run summaries and HTTP interaction transcripts—removing HTML boilerplate hints so agents cannot pass checks without real execution output.
- Enforce guardrails on automated refactoring: Gate checks should explicitly flag unexpected new top-level directories and shrinking test counts, closing common failure modes where agents simplify code by removing safety checks.
As engineering teams increasingly integrate agentic automation into their delivery pipelines, developers can try top AI models cheaply through one API at https://apixoai.online to build and benchmark custom pull request triage and review systems. By establishing strict, deterministic gates today, teams can safely scale up agent-driven contributions without sacrificing code quality or operational stability.
Source: Measuring Review in Human Minutes: A Risk-Tier Gate for Agent Pull Requests in .NET — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
