The Problem Hidden Inside “Convert It to Markdown”

anydoc is a Rust library and toolchain for turning Word, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV, and PDF files into GitHub-Flavored Markdown. This opening asks a narrower but more consequential question: what must happen between receiving a mixed bag of documents and producing text that a person or a downstream system can trust? The answer supports the central judgment of this series: anydoc is best understood as a semantic boundary between incompatible document formats and a common Markdown-oriented workflow.

That boundary is useful precisely because it is selective. It tries to preserve structure, relationships, and recoverable source data, while accepting that Markdown cannot reproduce every visual or interactive property of a document. To understand when the design fits, readers need a model of four things: how an input is identified, how source meaning is normalized, how a common output is rendered, and when the system must refuse to call the result complete.

The false simplicity of a mixed document set

Imagine a workflow that receives a recent Word report, a legacy spreadsheet, a slide deck, a CSV export, and a PDF. A human can open each file in its familiar application. A text pipeline cannot rely on that shared visual environment. It needs one representation that can be searched, indexed, summarized, or passed to another system without asking the downstream consumer to understand every original format.

“Convert everything to Markdown” sounds like a request for text extraction. But a useful result must answer questions that plain extraction leaves unresolved. Is a line a heading or merely large text? Is a number part of a table, a list, or a paragraph? Does a reference point to a note, a web address, or another place in the same document? Is an image content that should be retained, a URL that can be rendered, or an unavailable object whose alternative text is all that remains?

The project describes its best fit as a pipeline receiving mixed office documents and needing one consistent, structured Markdown output. Consistency here does not mean that every source becomes visually identical. It means that comparable structures are handled by the same output policy: headings remain headings, tables remain tabular where the target allows it, links retain their destinations, and notes or formulas do not depend on which office format carried them in.

This distinction matters because a converter can fail in two opposite ways. It can preserve too little, returning a flat text dump that is easy to generate but hard to interpret. Or it can promise a visual replica of the source, even though Markdown has no general way to express merged cells, arbitrary layout, embedded binary objects, or every numbering style. anydoc takes the middle path: normalize the semantics that its model can carry, then make the target format’s limits visible in the output and the API.

A converter is a boundary, not a single parser

The architecture follows that goal in stages. Document bytes first go through format detection. A format-specific frontend then interprets the source package or stream. For the office and text formats that use the common route, the frontend constructs a shared Document model. Finally, one Markdown serializer renders that model as GitHub-Flavored Markdown. The separation is important: the frontend absorbs differences in source syntax, while the renderer does not need a separate set of Markdown rules for every input family.

flowchart LR A["Mixed document bytes"] --> B["Content-based detection"] B --> C["Format-specific frontends"] C --> D["Shared Document model"] D --> E["One GFM Markdown serializer"] E --> F["Structured Markdown"] B -->|"PDF"| G["PDF inspector"] G --> F G -->|"Scanned pages"| H["Needs OCR"] H -->|"Opt-in hosted path"| I["Firecrawl Parse OCR"] I --> F

The shared model is the architectural hinge. It contains block-level structures such as headings, paragraphs, lists, tables, quotations, code, rules, and displayed formulas. Inline content can carry styles, links, images, anchors, note references, line breaks, formulas, and checkboxes. The point is not to expose every tag or record type from a source file. It is to give downstream code a vocabulary for the structures that matter after conversion.

That vocabulary also determines who owns difficult decisions. A frontend resolves source-specific style cascades, numbering, and references before handing content to the model. Lists arrive with their marker family and source numbering; links distinguish external, relative, and internal targets; notes are attached to document-scoped identifiers. Embedded assets remain available with their bytes and media types, so the model can outlive the archive from which it was built. The renderer receives resolved meaning instead of having to understand how Word, OpenDocument, or RTF happened to encode it.

PDF makes the boundary easier to see because it is an explicit exception. Text-based PDFs are converted through pdf-inspector, which emits Markdown directly. They do not pass through the shared document model, and the public API does not offer a model form for them. This is not necessarily an architectural flaw. It is a decision not to force a source with a different extraction path through a model that the project does not claim to support for PDF output.

What “consistent” means—and what it cannot mean

A single serializer makes cross-format behavior more predictable. A correction to table escaping, heading anchors, footnotes, or inline styles can apply wherever the shared model is used. The serializer also has to protect the structure it is producing. Source text may contain characters that look like Markdown syntax, so escaping depends on context: a pipe inside a table cell is different from a pipe in ordinary prose; a bracket inside a link label is different from one in a paragraph; a line beginning with a number may accidentally become a list. Treating these cases as mere string concatenation would allow source content to change the meaning of the generated document.

Tables show both the strength and the limit of normalization. anydoc represents a table as a canonical grid. A cell that spans rows or columns has one origin, while covered positions point back to it. This keeps columns from being duplicated when different source formats describe spans differently. But GFM tables do not have a general cell-span syntax. The covered positions therefore render as blank cells. The source geometry remains coherent enough for a Markdown table, but the original merged-cell appearance does not survive.

Images expose a related trade-off. An image with an external URL can become an ordinary Markdown image. An embedded image cannot be embedded as raw bytes in the Markdown text, so it renders as its alternative text while its bytes remain available on the document model. That is a loss in the default text representation, but not necessarily a loss of all recoverable information. A consumer that needs a richer asset treatment can inspect the model rather than pretending the Markdown alone contains the original object.

Lists, formulas, notes, and internal links reveal why structure matters more than a plain-text dump. The renderer can preserve source numbering where GitHub-Flavored Markdown supports it, turn equations into LaTeX forms, number note references consistently, and map internal targets to generated anchors. Where the target has no equivalent—for example, some non-decimal list markers—the output keeps the source marker visible as literal content rather than silently inventing a different numbering meaning.

The result is a useful definition of consistency: the same semantic policy is applied to every model-backed input, and the places where the target is weaker are handled deliberately. It is not a promise that a .doc and a .pptx will have the same visual layout, or that a Markdown consumer will recover every property of the source application.

A practical test for trustworthy output

A reader evaluating any converted document can begin with four questions. First, was the input identified correctly? Content-based markers are stronger than a filename, but formats without a signature, such as CSV, still need an extension or explicit format. Second, did the output preserve the structures that matter to the workflow—headings, tables, links, notes, lists, formulas, and useful asset metadata—rather than merely retaining words?

Third, is the result complete under the converter’s own contract? anydoc treats unsupported, encrypted, malformed, missing, and resource-limited inputs as typed failures when no complete useful Markdown can be produced. It does not make a scanned PDF look successful by returning text from only the pages that happen to be searchable. A later chapter will examine how recovery and fixed safety limits make that distinction operational.

Fourth, does the chosen path fit the operating conditions? The Rust core converts locally and does not perform OCR or network calls. Node, Python, and the CLI can opt into hosted OCR for documents that need it, which introduces a deliberate data-boundary and service dependency. The browser build, language bindings, and Agent Skill make the same core idea available in different environments, but they do not erase the differences between local conversion, hosted recovery, and access to the intermediate model.

The project’s benchmark gives useful but bounded evidence for this design. Its README reports a 4.4-millisecond median on a stated corpus and compares quality across fourteen benchmarked formats. The benchmark documentation says that PDFs are out of scope, the corpus is not redistributable, and speed and quality use specific warm-run and paired-judge methods. Those conditions matter: a project-reported result can show what the system is optimized to do without proving that every document collection or downstream Markdown consumer will behave the same way.

That is the central problem hidden inside the short command “convert it to Markdown.” The command names an output format, but a trustworthy system must make prior decisions about identity, structure, preservation, completeness, safety, and deployment. anydoc’s answer is to place a shared semantic boundary between source-specific frontends and a common renderer, then document the places where that boundary stops. The next question is how the first decision—identifying the input—can be made from evidence in the bytes, and how the selected frontend turns that identity into comparable meaning.

References