The First Constraint Is Seeing the Page
Granite Vision's visual design addresses a specific problem: how to give a compact language model enough trustworthy detail from a document to answer questions about it. This chapter focuses on the path from a page image to language-compatible visual evidence. Understanding that path helps a reader judge why Granite Vision uses global and tiled views, why it keeps features from multiple visual layers, and where those choices create costs or leave important behavior unmeasured.
Documents force a choice about resolution
A natural image often remains recognizable when it is reduced. A photograph of a car can still show a car after losing some fine detail. A document is less forgiving. The meaning of a small word, a decimal point, a table border, or a chart legend can determine the answer to a question. Shrinking the page may preserve its overall appearance while destroying the evidence that separates a correct answer from a plausible one.
This is the difference between seeing a page and reading it. Visual document understanding requires the system to preserve content and relationships at the same time. It must notice the characters in a field, but also know which label governs that field. It must recognize a number in a table, but retain its row and column context. It must identify a mark in a chart, but connect it to the right series and scale. A single low-resolution view can make these relationships unavailable before the language model has any opportunity to reason about them.
Granite Vision uses SigLIP as its pre-trained visual encoder with a base resolution of 384 × 384 pixels. That base size gives the encoder a consistent input unit, but it does not mean that every document is forced into one square view. The paper describes an AnyRes-style approach that combines a global image with a grid of additional views. The system selects a nearby resolution from a predefined set, divides the page into 384 × 384 patches, and can use as many as ten patches. The available tiling arrangements cover very different shapes, from tall pages to wide layouts.
The global view and the patches serve different purposes. The global view preserves the page's large-scale organization. It can help locate a title, a table, a figure, a footer, or a cluster of form fields relative to the whole document. The patches provide more pixels for local regions, making small text and fine graphical details easier for the encoder to represent. A tall form, for example, may need several vertically arranged views so that a name near the top and a signature area near the bottom remain legible without losing their positions on the page. A wide financial chart may need a different arrangement so that the plot, legend, and labels receive adequate coverage.
This is not the same as simply enlarging an image. Enlargement makes more pixels available in the file, but it does not automatically create a useful way for the model to process them. Granite Vision's tiling choice establishes multiple visual observations and preserves a relation between those observations and the original page. The system can inspect a local region closely while retaining a page-level reference for interpreting what that region means.
The choice of tiling shape matters because documents do not share one geometry. A portrait form, a landscape spreadsheet, and a page containing a wide chart place their important evidence in different arrangements. If every page were forced into the same square, one dimension could become needlessly compressed or a relevant region could occupy too few pixels. The paper reports a set of resolution and tiling options rather than one fixed crop, allowing the visual input to adapt to the page's aspect ratio. That flexibility is useful precisely because the model's question may concern a small region whose importance is not apparent from the page's overall shape.
The global view also limits a common mistake in patch-based processing: treating each crop as an independent image. A patch can make a word clearer while removing the heading that explains it, or show a number while hiding the table row that gives it meaning. The whole-page view supplies the missing reference. Conversely, the global view can preserve the hierarchy while making the characters too small to read. Granite Vision's design therefore creates a division of labor between views rather than asking one view to be both a map and a magnifying glass.
There is an important qualification here. The paper describes the available patch budget and the range of tiling arrangements, but it does not say that the system always selects the ideal view for every page or question. A crowded page may contain more small regions than the selected views can represent clearly. A fold, scan artifact, unusual font, or overlapping annotation may still defeat the encoder. The visual path improves the chance that decisive evidence reaches the language model; it does not make the page lossless. An enterprise system would still need to measure errors on the documents it receives, especially where a missed detail has a material consequence.
The distinction becomes concrete in a receipt. Suppose the image contains several line items, a subtotal, a tax, and a final amount. A coarse view may make every amount look like readable text but fail to preserve the alignment that tells the model which amount is the total. A set of local views can make the relevant digits clearer, while the global view helps identify where the total appears in the document's hierarchy. The architecture does not guarantee that the answer will be right; it makes the necessary evidence more available to the later stages.
Charts expose another failure mode. A chart can contain a title, multiple series, a legend, axis labels, tick marks, and annotations. A small colored marker may be visually important even though it occupies very few pixels. If the system sees only a broad pattern, it may describe the chart's trend but miss the exact series or value named in the question. Preserving local detail is therefore tied to reasoning: the model cannot compare the requested quantities if the marks, labels, or scale have already been blurred together.
Granite Vision also extracts and concatenates features from multiple layers of the visual encoder. The paper's reason is that different layers can provide complementary levels of representation that are useful for fine-grained document understanding. It does not assign a simple rule such as “one layer reads text while another reads layout.” The safer interpretation is that the combined representation gives the next component access to more than one level of visual information. A document question often needs both: the identity of a small character and the structure that gives that character its meaning.
That design introduces a resource trade-off. More views and richer features can preserve more evidence, but they also create more visual information for the rest of the system to process. The paper specifies the resolution strategy and patch budget, but it does not provide a complete inference profile for latency, memory use, or energy on a target enterprise hardware stack. Nor does it present an ablation that isolates the contribution of every tiling option or every encoder layer. The mechanism is well described; the operational price and the independent contribution of each choice remain questions for measurement.
From pixels to language-compatible evidence
The visual encoder does not produce a finished answer. It produces features: an internal representation of the image that a language model cannot use directly unless the two systems share a suitable interface. In Granite Vision, the encoder's output is passed through a projector, a two-layer multilayer perceptron with GELU activation. The projector converts the visual features into a sequence of visual tokens with the same dimensionality as the language model's word-embedding space.
The word “token” can sound more mysterious than it is. A language model receives a sentence as a sequence of units that represent pieces of text. Granite Vision creates a corresponding sequence for visual information. These visual tokens are not a transcript of the page and are not described in the paper as a separate list of recognized words. They are the form in which the model's visual evidence can enter the same downstream sequence as the user's instruction.
The interface has a clear division of responsibility. The visual encoder maps the image into features. The projector changes their representation so that the language model can consume them. A language encoder tokenizes the instruction. The visual and language tokens are then concatenated and fed to a decoder-only language model, Granite 3.1-2B-Instruct, which generates the response one token at a time. In simplified form, the process is: image to visual features, visual features to visual tokens, instruction to language tokens, both sequences into the language model, and language model output to an answer.
The sequence matters because each boundary changes what information is available. Before encoding, the page is a two-dimensional image. After visual encoding, it is a sequence of learned features. After projection, those features have the dimensional form needed to sit alongside language tokens. Once the sequences are concatenated, the language model can use the instruction to determine which visual evidence is relevant to the response. The model is not first producing a complete visual report and then asking a separate text system to search that report; the paper describes one language model receiving both kinds of tokens as input to generation.
That arrangement also clarifies why the question itself can change the required visual work. A request to describe a form may reward a broad account of its purpose. A request to name the member or detect a missing signature requires attention to a field and its state. A chart-extraction request requires preserving a structured relationship between marks and values. The same page can therefore lead to different useful answers depending on the language tokens that direct the model. The visual representation must be sufficiently rich for those different instructions to select different evidence.
This division matters because a projector is not a magic bridge that solves every mismatch. It does not decide which table cell is correct by itself, nor does it supply an independent guarantee of optical character recognition. Its job is to make the visual representation usable by the language model. Whether the combined system answers correctly depends on the quality of the visual evidence, the learned alignment between that evidence and language, and the task behavior learned during training.
A question supplies the other half of the interface. “What is the highest value inside the chart under Overall Accuracy?” is not equivalent to “Describe this image.” The first instruction directs the model to locate a particular region, identify the relevant measure, and return a selected value. Because the instruction tokens and visual tokens reach the same language model, the model can condition its response on both the page and the question. That shared sequence is what turns visual access into question answering rather than image description alone.
The architecture also explains why document-specific visual evidence cannot be replaced by a longer text context. A language model may have a large capacity for text, and the paper describes Granite 3.1 as supporting up to 128K tokens of context. But a large language context does not automatically mean that a visual model can process arbitrarily long or multi-page documents. Each page still has to be represented, and the relevant details must survive the visual encoding and token interface. The paper names multi-page processing through context compression as future work, which is a direct warning against reading the language-model context figure as a demonstrated multi-page capability.
There are several ways this visual path can fail. If the selected views do not preserve a small field, the language model cannot recover evidence that never reached its input. If a patch contains a number without enough surrounding layout, the model may read the characters but assign them to the wrong row or label. If the projector or later reasoning process fails to align visual tokens with the instruction, a clear page can still yield an irrelevant answer. And if the answer is fluent, that fluency can make the failure harder to notice. The architecture reduces some information bottlenecks; it does not remove the need for task-specific training and evaluation.
The path also makes compactness a more precise engineering question. Granite Vision is not using a large language backbone to compensate indiscriminately for a weak visual interface. It puts effort into how visual evidence is selected and represented before the 2B language model receives it. This may be a sensible allocation for enterprise documents, where the limiting factor is often not a lack of general conversational ability but the loss of a small, decisive detail. Still, the paper presents the components as a coordinated design rather than proving through a complete ablation that one part would fail without the others.
The most important conclusion is therefore conditional. Granite Vision's high-resolution, multi-view visual path is designed to keep both global structure and local detail available to the language model. Multi-layer features broaden the representation, and the projector makes that representation compatible with language tokens. These choices explain how the system can attempt document questions; they do not by themselves establish how fast, how reliable, or how economical it will be on a particular document stream. The next boundary is supervision: the model needs training examples that teach it what to do with the evidence it can see, including how to read tables, charts, forms, and diagrams rather than merely describe their appearance.