Turning page geometry into readable Markdown

For pdf-inspector, trustworthy characters are only the raw material. This chapter focuses on the next problem: how to turn positioned text and drawing geometry into reading order, tables, headings, and Markdown without mistaking a chart for a table or a two-column page for one interleaved paragraph. The reader's practical goal is to see layout-aware extraction as guarded inference, where every convenient interpretation must earn its place through spatial evidence.

Why reading order is an inference problem

A simple top-to-bottom sort works for a single column whose lines were emitted in visual order. It fails on a newspaper page or a research paper with two columns. The first line of the left column may sit at almost the same height as the first line of the right column, so a global y-sort alternates between them. The output can contain all the right words and still destroy the document's argument.

pdf-inspector starts with the positions it retained during native extraction. For each page, it builds a horizontal occupancy profile: a compact picture of where text occupies the page's width. A sustained low-occupancy valley can indicate the gutter between columns. The detector then checks whether the proposed regions have enough items, span a meaningful vertical range, and contain evidence on both sides. A gap at the page margin is more likely to be empty space than a column boundary; a gap that appears only among a few scattered labels is more likely to be a form or a table than flowing prose.

The key is that a gutter is not sufficient by itself. A table can have clean vertical gaps between its fields. A checklist can place short labels in regular bands. A sidebar can sit far from the main text while containing too little material to behave like a full column. pdf-inspector uses prose-density checks to distinguish these cases. It groups items into rough lines, measures how much of each proposed column those lines fill, considers the continuity of full-width lines, and checks the average number of items per line. Flowing prose tends to produce sustained lines across a column; a grid of short cells tends to produce many small, scattered items.

Full-width content creates another trap. A title or introductory paragraph can cross the gutter even when the body below uses two columns. If that wide item were allowed to fill the occupancy profile, it could erase the evidence for the split. pdf-inspector excludes sufficiently wide spans from the column signal, then uses the remaining page evidence to infer the local layout. This does not mean the title disappears. It means the title is kept as content while being prevented from deciding a question it cannot answer.

When the histogram is ambiguous, pdf-inspector has a bounded horizontal-gap fallback inspired by XY-cut methods. It looks for a substantial gap between item extents, requires enough items on the major and minor sides, and checks that the two sides overlap vertically. This helps with asymmetric layouts such as a main article beside a narrow sidebar, where the sidebar may be too sparse to create a strong occupancy valley. It is deliberately a simplified, single-level fallback rather than a claim to implement a full recursive page segmentation algorithm.

The detector also has to know when not to split. A table-aware guard prevents a table's geometry from becoming a false newspaper layout. On pages containing charts, chart regions can be separated from surrounding prose so a figure does not reorder the text above and below it. The result is a set of hypotheses with rejection conditions, not a universal promise that every page has one correct column model.