Keep Long Reviews Inside Their Limits

OpenCodeReview’s bounded review units solve one problem but create another: a valid group can still be too large, a tool conversation can still grow too long, and a whole pull request can still spend more tokens than a team intended. This chapter explains how OCR separates those resource pressures and keeps them from being mistaken for one generic “max tokens” setting. The reader should leave able to tell whether a file was too large to start, a conversation was compressed, or a run stopped after its aggregate budget was reached.

These controls are part of review correctness. If a system sends an oversized prompt and receives a truncated answer, it should not present the result as equivalent to a complete review. If it stops dispatching groups because a run budget is exhausted, it should not silently convert the undispatched work into a clean result. OCR’s resource policy is therefore also a language for describing what the system did not do.

Three budgets answer three different questions

The first distinction is between input and output. OCR’s prompt limit bounds the context sent for a review unit—a file group in diff review. The completion limit bounds what the model can return. The current review template sets a 200,000-token prompt ceiling and a separate 16,384-token completion cap. Raising the prompt ceiling for a model with a larger context window does not automatically authorize a larger answer. The two limits protect different resources and should be tuned for different reasons.

A third control applies across the whole run: the aggregate token budget. It counts the input and output usage of the review’s LLM calls and can stop OCR from scheduling later groups once the configured cap would be exceeded. Groups already in flight are allowed to finish, so the final spend can exceed the exact look-ahead estimate by the bounded amount of concurrent work. The resulting run can be partial: some groups completed, while later groups were deliberately not dispatched.

That distinction changes practical cost reasoning. Suppose a change has six small groups. A per-group prompt limit may allow every group to fit, but a whole-run budget may allow only four of them at the chosen effort. Conversely, a high aggregate budget does not make one enormous group safe if its prompt cannot fit the model context. A team needs both local fit and global spending control.

The per-file and per-group gates are intentionally conservative. OCR excludes a raw diff when it exceeds 80% of the prompt limit before grouping, because the rendered prompt will need room for rules, instructions, context, and tool interaction. It checks the final group prompt again before the main request. A group that crosses the threshold can stop without an LLM call, and grouping can split a proposed group whose combined diffs are too large. The system is refusing to pay for a request whose evidence cannot fit comfortably, not claiming that the underlying code is safe.