Before Conversion Begins: How anydoc Decides What a Document Is
anydoc’s first hard problem is not producing Markdown. It is deciding what the incoming bytes are allowed to mean. This chapter focuses on the boundary between format detection and actual parsing: how anydoc uses evidence from a file or byte stream to choose a frontend, and how that frontend turns source-specific semantics into the shared representation that later rendering depends on. Understanding that boundary lets readers distinguish a correct format guess from a successful conversion.
Detection is a claim about identity, not a conversion result
A filename is a useful hint, but it is not a reliable description of a file’s contents. A document can be renamed, exported with an unexpected extension, or wrapped in a container whose internal metadata is more authoritative than its name. anydoc therefore begins, where possible, with content-based detection. The detector does not ask whether the document’s prose “looks like” a report or a spreadsheet. It looks for the identity markers that the relevant format specification assigns to the container.
For a PDF, that marker is the %PDF- header. An RTF file begins with its characteristic opening group. Legacy Office formats use OLE compound storage, so anydoc looks for the mandated stream names that distinguish Word, PowerPoint, and Excel. Modern Office packages, OpenDocument files, and EPUBs are ZIP-based, but a ZIP file by itself is not a document identity. The detector examines package-level evidence such as the ODF or EPUB mimetype, the main-document relationship, content types, and, when metadata is stale or generic, the main part’s root element.
This is a conservative sequence. A ZIP archive that merely contains a PDF is not classified as a PDF if the package structure identifies it as a Word document. A plain ZIP archive with a text file is not promoted to a document format simply because it can be opened. When the detector cannot establish a trustworthy identity, it returns no format rather than guessing. That leaves the caller’s fallback and the eventual conversion error to explain what remains unknown.
The layered checks inside a package are also a useful design lesson. A package-level relationship can identify which part is the main document; its content type can identify the document family; and the root element can serve as a further authority when the metadata is stale or generic. If those clues are unavailable, conventional part locations provide a bounded fallback rather than a broad search through every file in the archive. This lets the detector tolerate ordinary variation in producers without turning “anything that contains something familiar” into a format rule.
The same caution applies to legacy binary files. The outer OLE signature only says that the bytes are a compound file. The required stream name supplies the more specific identity. That extra step matters because several old Office formats share the same container technology. Identity comes from the relationship between the container and its designated content, not from the container label alone.
The distinction between detection and parsing is essential. Detection answers, “Which family of rules should inspect these bytes?” Parsing answers, “Can those rules extract a meaningful document from them?” A package can have a recognizable Office identity and still be encrypted, malformed, missing a required part, or too large to process safely. Treating identity as proof of convertibility would make the first step look more successful than it is and would encourage callers to ignore the later failure surface.
Path-based conversion uses a sensible fallback. If content detection succeeds, it takes precedence. If the bytes have no recognizable signature, the path’s extension can name the format. Extension matching is case-insensitive, and the public API also allows callers to supply a format explicitly when working with in-memory data. The fallback is not a replacement for content evidence; it is a way to handle the cases where the format itself provides no usable signature or where the caller already knows information that the byte stream cannot carry.
That distinction is especially practical for ingestion systems that do not preserve original filenames. A service may receive a byte buffer from an upload, a queue, or another API and still want content-based detection for signed containers. If the buffer is CSV, the service must carry the format as metadata or choose it from a trusted caller decision. In other words, the explicit format argument is not just an escape hatch for unusual files; it is the correct way to supply missing identity information when the transport has stripped away the path.
The choice also affects error handling. A caller that passes an explicit format is asking anydoc to interpret the bytes with that frontend; it is not asking the library to pretend that a malformed result is valid. The selected parser can still reject the input, and the resulting error retains a different meaning from “no format was identified.” Keeping those states separate makes batch logs and retry policies more useful.
Why signatures beat filenames—and why CSV is different
Specification-level markers are valuable because they are tied to the container’s identity, not to an accidental label. A mislabeled OOXML package can still be routed by its relationships and content types. A legacy binary workbook can still be distinguished from a Word file by the stream it contains. This improves the chance that the right parser is selected before any source-specific assumptions are made.
The design also avoids an attractive but dangerous alternative: guessing from document content. A paragraph that contains comma-separated values does not prove that the file is CSV. A ZIP archive with an XML part does not prove which XML vocabulary it contains. Heuristics can turn an ambiguous input into a confident but wrong parser choice, producing output that looks plausible while silently discarding structure. anydoc’s detector instead treats ambiguity as a reason to defer or fail.
CSV exposes the limit of this method. CSV is plain text and has no container signature that can identify it unambiguously. The same bytes could be a delimiter-separated table, a fragment of text, or data intended for another consumer. Consequently, anydoc does not detect CSV from bytes alone. A caller must name the format through an extension, path, or explicit format argument. That is not an arbitrary inconvenience; it is a consequence of asking the input itself to provide evidence it does not contain.
The same principle explains why detection can return None without raising a detailed parsing error. At this stage the system has not yet selected a trustworthy interpretation. A later caller may have an extension that resolves the ambiguity, or may decide that the input should be rejected. Separating “not identified” from “identified but structurally unusable” gives those cases different meanings and lets an ingestion pipeline record them accurately.
The diagram’s last branch is the important one. Even a correct parser selection does not guarantee a model. The frontend still has to read the package, resolve its relationships, and decide whether enough meaningful content remains. Detection is the gate to an interpretation, not a promise about the interpretation’s outcome.
The frontend’s real job is semantic resolution
Once a frontend is selected, its work is larger than translating tags into generic text. Different document systems encode similar ideas in different ways. A style may be inherited through a cascade. A list number may come from a numbering definition rather than the visible text. A link may target a bookmark inside the document, a relative resource, or an external URL. An image may be stored as a package part whose bytes must remain available after parsing. If each of those choices were postponed until Markdown rendering, the common renderer would need to understand every source format’s inheritance and reference rules.
anydoc places that responsibility at the frontend boundary. The model documentation describes styles, numbering, and references as resolved before content is constructed. A style in the model has definite flags such as bold, italic, strike-through, or code rather than an unresolved “maybe inherited” state. A list carries its marker family, its starting ordinal, and, when a source number is too composite to reconstruct from position alone, literal marker text. These values describe what the source means at the point where the renderer needs to use them.
The same rule applies to relationships. The shared model can represent a document-scoped anchor, an internal link target, an external URL, a relative reference, a note reference, or an image whose bytes are retained as an asset. The frontend has already done the source-specific work of finding those relationships. Downstream code can reason about “an internal target” without knowing whether the source called it a bookmark, a named destination, an EPUB fragment, or something else.
The resulting Document is self-contained enough to outlive the source archive. It has body blocks, notes, and embedded assets; assets carry their bytes, media types, and origin information. That does not mean the model preserves every visual feature of the original application. It means that the data represented by the model is explicit and portable. A renderer can choose how to express it, and a consumer can inspect retained assets when Markdown alone is not enough.
This is also where ownership becomes visible. The frontend owns questions about source identity, relationships, inherited meaning, and incomplete source parts. The model owns a stable description of the resolved result. The consumer should not need to reopen the original archive to discover whether a link was internal or whether an asset came from an embedded package part. Keeping those responsibilities apart makes the later renderer simpler, but it also means that a frontend that resolves a source feature incorrectly can affect every output surface that shares the model.
The block and inline vocabulary is deliberately structural. Blocks include headings, paragraphs, lists, tables, quotations, code, rules, and displayed formulas. Inline values can carry styled text, links, images, anchors, note references, line breaks, formulas, and checkboxes. This is enough to preserve relationships that matter to a text workflow without exposing the XML namespaces, binary records, or package paths used to encode them.
That boundary creates a useful division of labor. Format frontends are responsible for understanding source semantics and for deciding what can be resolved. The shared model is responsible for carrying those resolved semantics consistently. The Markdown renderer is responsible for expressing them safely in the target syntax. When a source feature has no meaningful place in the model, the limitation belongs to the conversion boundary; when the model contains the feature but Markdown cannot express it fully, the limitation belongs to the renderer or output format.
This division also explains why a common model is a form of risk control. It prevents every consumer from reimplementing source-specific parsing and prevents every renderer from inventing its own interpretation of styles, numbering, or links. But it does not remove the need for judgment. A model can preserve an asset while the default Markdown output shows only its alternative text. It can preserve a merged table’s grid coordinates while GFM renders covered cells as blanks. Normalization makes the loss deliberate and inspectable; it does not make the target format more expressive than it is.
The practical conclusion is simple: when anydoc identifies a file, it has only chosen the language in which the next question will be asked. The frontend must still resolve that language into a common set of structures and relationships. Only then can one renderer give different source formats a shared Markdown behavior without collapsing them into unstructured text. The next step is to examine that renderer, where the model’s promises meet the exact rules and limitations of GitHub-Flavored Markdown.