A quality-aware playbook for PDF ingestion

The central question about Firecrawl's pdf-inspector is not whether it can emit Markdown. It is whether a production system can know when that Markdown is supported by the PDF, when the page needs visual recovery, and when neither local path justifies a claim of completeness. This final chapter turns the project's mechanisms into a page-aware operating policy. The goal is a defensible choice among native extraction, selective OCR, fusion, and hosted escalation, with enough provenance to explain the choice afterward.

The unified model: preserve structure and uncertainty

pdf-inspector is best understood as one chain of evidence rather than as a stack of unrelated features. It begins with a PDF representation that may contain text operations, font-local codes, images, paths, annotations, and layout state. Native extraction walks that representation and preserves positions, font information, drawing geometry, and page relationships. Quality analysis then asks whether the recovered characters and visible coverage are credible. Layout reconstruction turns credible geometry into lines, columns, tables, and Markdown. Finally, page-level signals decide whether the local native result is enough or whether visual recovery should begin.

The chain has two preservation rules. First, preserve structure: a character is more useful when its position, baseline, style, neighboring items, and relationship to a table or column remain available. Second, preserve uncertainty: a suspicious mapping, estimated advance, OCR confidence, warning, or incomplete page should not disappear when the output is formatted. Structure without uncertainty can make a wrong result look authoritative. Uncertainty without structure makes it difficult to recover the missing meaning. pdf-inspector's design connects them so a later stage can make a better decision than a flattened string would allow.

This model also clarifies the project's relationships. Classification does not replace extraction; it decides where extraction is likely to be useful. Extraction does not replace layout; it supplies the positioned evidence layout needs. Layout does not prove coverage; it organizes what native parsing recovered. OCR does not replace either path by default; it supplies a second observation for pages whose native evidence is missing, garbled, or visually incomplete. Fusion does not conceal disagreement; it compares candidates and records which source contributed to the result.

A production owner should ask the same questions at each boundary. What crossed the boundary: bytes, positions, page numbers, rendered pixels, OCR polygons, model identity, or Markdown? Which component owned the relevant state? What triggered the transition? What would happen if the next component returned no result, a reordered page, an invalid coordinate, a weak confidence, or an unavailable runtime? The answers turn a feature description into an operating model.

A page-aware production policy

A useful policy begins by treating the document as a collection of pages with related but nonidentical evidence. The detector can classify a document as text-based, scanned, image-based, or mixed, report a confidence score, and identify pages needing OCR. That is a starting point for routing, not a final quality verdict. A mixed document should not inherit the strongest page's status or the weakest page's cost by default.

For a report with ordinary text pages, a scan-heavy appendix, and a form-like cover, the first pass should preserve the distinctions. Native extraction can continue for pages whose text and coverage signals are credible. Pages with a full-page image, vectorized lettering, empty text, or suspicious encoding can be routed to visual recovery. A page with a difficult table but trustworthy text may need better layout handling rather than OCR. A page with a clean overlay on a scan needs a coverage decision rather than a character-decoding decision. The policy is page-aware because the causes are page-aware.

Detection strategy is part of that policy. The current implementation uses an evenly distributed sample of up to eight pages by default, then performs additional analysis for page routing. Sampling is a useful compromise for large documents, but it can miss variation between sampled pages. A caller that needs a more accurate mixed-versus-scanned judgment can choose full scanning, while a caller that already knows which pages matter can provide an explicit selection. The repository also contains a summary that describes early-exit scanning as the default, so exact configuration claims should follow the current implementation rather than silently harmonizing the discrepancy.

The next choice is how much native work the application needs. Detection-only is appropriate when a service must make a quick routing decision. Analysis is useful when positions and layout complexity matter but Markdown is not yet the desired output. Full processing is appropriate when the application wants native Markdown and its page-level metadata. The high-level path loads the document once and shares it between detection and extraction, while per-page results provide a better boundary for retries and selective escalation than one concatenated string.

For OCR, Off is the conservative native-only policy. It returns native results and quality metadata without entering the vision runtime. Auto is the default policy for a hybrid pipeline: it intersects recommended pages with any caller-selected pages and routes only the resulting set. Force is an explicit comparative or uniform-processing policy that recognizes every selected page, including pages with clean native text.