The paradox of better tools: When an upgrade makes results worse

The GitHub Copilot code review team faces a seemingly simple task: an AI agent receives a pull request, explores the changes, searches the surrounding code, and finds potential issues. Originally, it used its own set of tools — list_dir, search_file, search_dir, and read_code — designed specifically for code review. But in parallel, Copilot CLI existed, using shared tools grep, glob, and view for code exploration. They were better maintained, more thoroughly tested, and used by other Copilot products including the cloud agent. Unifying the infrastructure and eliminating duplicate implementations made perfect engineering sense. The test results, however, pleased no one: average review costs rose and the number of useful comments dropped. Better tools led to worse results.

The problem wasn't the tools — it was the agent's head

The key to the mystery was revealed only by detailed benchmark traces — recordings of how the agent proceeded step by step. It turned out the agent wasn't behaving like a code reviewer, but like a programmer exploring an unfamiliar repository. Instead of starting from the pull request diff and asking targeted questions ("Where is this function called?", "Is there a test with the same pattern?"), the agent aimlessly browsed the directory structure, guessed file paths, opened large blocks of code, and accumulated context it didn't need for the review at all. This behavior wasn't a model flaw nor a failure of the grep, glob, and view tools. It was a consequence of the instructions telling the agent how to use the tools. Instructions originally designed for Copilot CLI — where exploring the entire repository makes sense — weren't suitable for the narrowly focused task of code review. This insight is crucial for AI agent development: tool descriptions and agent instructions are not an implementation detail. They are API documentation for the LLM. Just as unclear API documentation leads a human developer to inefficient decisions, unclear instructions lead an AI agent to waste tokens and produce worse results.

How GitHub rewrote the reviewer's brain

The team led by engineer Napalys Klicius undertook a radical rewrite of the instructions. The new workflow reflected how a human reviewer actually works: 1. Start from the diff. Instead of general repository exploration, the agent first formulates specific questions arising from the pull request changes. 2. Narrow first, then read. The agent first uses grep to search for symbols and calling functions, and glob to find relevant files. Only when it knows exactly what it's looking for does it open specific line ranges using view. 3. Batch independent searches. Instead of alternating one search and one read, the agent performs several independent grep/glob queries at once — and only then proceeds to targeted reading. 4. When a search fails, fix it — don't expand. If grep finds nothing, the agent tries a simpler escaped expression. If the path is wrong, it uses glob — it doesn't guess neighboring paths or open random files. This change in instructions — just a few extra sentences — reduced average review costs by roughly 20%, GitHub confirmed in the June changelog. Review quality remained preserved.

Why this story matters

GitHub's experience has broader implications for everyone building AI agents — whether code review agents, autonomous coding assistants, or agents for business process automation. Tools aren't neutral. The way you describe a tool to an agent fundamentally influences how it will use it. Instructions for grep like "search the code in the repository" lead to different behavior than "search the code related to this change in the diff." Benchmarks must show the path, not just the score. GitHub didn't just compare final numbers — they looked at what the agent was actually doing step by step. It was precisely trace analysis that revealed the agent was "browsing" instead of "investigating." Same tools, different tasks = different instructions. GitHub explicitly states they tried applying the same approach to Copilot CLI, but it didn't bring similar improvement there. In CLI, exploration makes sense — the developer can change direction, context evolves. Each product needs instructions tailored to its specific task.

Copilot Code Review: What it can do and how much it costs

GitHub Copilot code review is now available across the ecosystem — it works directly on GitHub (via the Request review button), in VS Code, Visual Studio, JetBrains IDEs, Xcode, and via GitHub CLI. A review typically takes less than 30 seconds, and beyond comments, Copilot also offers code fix suggestions that can be accepted with a single click. Regarding pricing plans, code review is available starting from the Copilot Pro plan at $10 per month (approx. 230 CZK). For teams, it's available within Copilot Business (from $19/month/user) and Copilot Enterprise (from $39/month/user). Organizations can also newly enable code review for members without a Copilot license — costs are then billed directly to the organization via GitHub AI Credits. For comparison: alternatives like CodeRabbit (from $12/month) or Amazon Q Developer (free tier for individuals, professional tier from $20/month) offer similar functionality. Copilot's advantage, however, is deep integration directly into GitHub — you don't need to install any additional app or manage external access tokens. As for Czech language availability — Copilot code review doesn't have full Czech localization, but it generates pull request comments in English by default. Using custom instructions in the .github/copilot-instructions.md file, however, you can enforce Czech or any other language. Just add the line "When performing a code review, respond in Czech." to the file, and Copilot will comment in Czech from then on. For Czech development teams that prefer communication in their native language, this is a simple yet effective solution.

What to take away from this

The story of GitHub Copilot code review is simple at its core: give the agent better tools, but also better instructions on how to use them. In the era of agentic AI, where autonomous systems are entering production workflows, this is a lesson that extends beyond a single company or a single product. For Czech developers and companies considering deploying AI into code review, the practical takeaway is: don't just rely on the AI tool "somehow" working. Invest time in custom instructions — define what the reviewer should focus on, what conventions your team follows, and what language it should communicate in. It's in these details that the difference is born between an AI assistant that just generates noise and an AI colleague that genuinely helps.