From Repository to Durable Page Queue

For OpenWiki, the answer to stale documentation begins with a state machine, not with a larger prompt. Repository generation turns a proposed wiki into an ordered set of page jobs, records which work is durable, and refuses to confuse an interrupted or source-changing run with a finished one. Understanding this lifecycle lets a reader trace an init or update, see where native and host-driven execution share the same guarantees, and judge why OpenWiki treats a page as a recovery unit rather than waiting for the whole wiki to be complete.

The six operations are a contract

Repository generation has six transport-neutral operations. They are implemented in the generation core and used by both OpenWiki's native runner and its host-agent integration. The names differ at the boundary—native code calls functions such as beginRepositoryRun, while an MCP host calls openwiki_begin—but the durable meaning is the same.

1. Begin establishes the run

beginRepositoryRun starts a fresh run or reconstructs one already recorded on disk. Before it touches the repository, it resolves the requested documentation language. An unrecognized language is rejected rather than silently converted to English, because changing the language after state has been written could leave an interrupted run that cannot be corrected cleanly.

After that gate, OpenWiki prepares the repository wiki, loads its ignore boundary, and checks for openwiki/.run.json. A fresh init prepares a replacement wiki while retaining a rollback path until the new run state and interrupted metadata are durable. A fresh update keeps the existing wiki and evaluates whether any work is needed. If a valid checkpoint exists, begin checks that the requested mode and language still match the durable run and reconstructs the process-local runtime.

The result is either an active run view or a proven no-op. This distinction is important. “No-op” is not a failure to create work; it is a deliberate conclusion that the repository, the existing evidence state, and the page-coverage record do not require a model invocation under the current request.

2. Submit the plan installs ordered work

During the planning phase, a model proposes pages and, for an update, any pages to delete. submitRepositoryPlan turns that flexible proposal into a normalized queue. Each page receives a fresh job identifier, a canonical path, a title, a purpose, research starting points, related pages, and relevant instructions. The job begins as pending.

The normalization step is where conversational intent becomes enforceable work. Paths are canonicalized and duplicate pages are rejected. A page cannot be both generated and deleted. Reserved working pages are refused. An initial repository plan must include the quickstart page and cannot delete generated pages; an update can contain explicit deletions, but not the canonical quickstart page. The queue is sorted deterministically, with quickstart placed after domain pages so navigation is generated from the structure that the run has actually produced.

The planner also cannot make known obligations disappear by omission. On update, OpenWiki inserts page jobs for pages carrying stale or unresolved grounding issues, and for pages that must be rewritten after a language change. The inserted job is not an automatic verdict that a fact is false. It is a requirement to inspect current source and make an explicit decision.

A repeated plan submission is safe only when it has the same semantic content. OpenWiki ignores generated job identifiers and progress when comparing a duplicate, but it rejects a different plan once a queue has been installed. That rule protects the meaning of the checkpoint: a caller cannot silently replace the work that an existing run says it is performing.

3. Next page exposes one current responsibility

nextRepositoryPage returns the first pending job. It does not reserve or mutate the job, so asking again is safe. The response includes whether the Markdown page already exists, its current Claim count, and the Claims that carry a stale or unresolved issue. Issue-free Claims are not repeated in the assignment; they can be retained deterministically.

The optional inspectRepositoryPageClaims operation is deliberately narrower than a general database read. It returns the complete current Claim set for the current pending page, without opaque evidence versions, and rejects a request for any other job. A worker needs this only when it intends to revise or remove content whose Claims are otherwise current and therefore were not included in the focused assignment. It is a read inside the generating phase, not a separate lifecycle stage.

This design gives the page worker a bounded responsibility. The native runner creates a fresh, non-delegating worker for the page, while a host integration asks its coding agent to research and author that same page using native repository tools. Neither driver gets to skip the queue or complete another page out of order. OpenWiki's core remains responsible for deciding which page is current and what completion means.

4. Submit page crosses the durability boundary

submitRepositoryPage is the point at which a page can become durable work rather than a model's temporary output. The current worker must have written readable Markdown. OpenWiki then repairs and validates the page's front matter, reconciles the page's Claim decisions, persists the Claim state, and proves that the persisted state agrees with the page bytes before advancing the queue.

Only the current pending job may be submitted. An already completed job can be acknowledged idempotently, which is useful when a caller retries after a response was lost, but an unknown job or an out-of-order submission is rejected. Correctable input problems—such as invalid page metadata or a malformed Claim payload—are returned in a way that lets the worker fix the page and try again. A failure that signals a broken durable invariant is not treated as an ordinary authoring mistake.

Once the page passes its proof, the queue records it as complete and records the producer that completed it. This is the key recovery decision in the lifecycle: a page that is complete is a self-contained unit of progress. The entire wiki does not need to be regenerated merely because a later page worker fails.

5. Finish closes the run, but only after the queue is drained

finishRepositoryRun refuses to run while any job remains pending. It may handle skipped jobs when their original snapshots are available, but it cannot report a clean completion while required work is simply unaccounted for. Finish applies planned and abandoned-page deletions, runs deterministic wiki finalization, restores skipped pages, completes the whole-run Claims proof, rebuilds page coverage, writes update metadata, and removes .run.json last.

The order matters because each operation depends on the previous state. A run-state file removed too early would erase the ability to retry after a finalization failure. A final index built before all domain pages exist could describe the wrong wiki. A durability proof run before skipped content is restored could mistake an intentionally deferred page for a missing page. Finish is therefore less a final model response than a controlled commit sequence.

stateDiagram-v2 [*] --> planning planning --> generating: submit_plan installs queue generating --> pending: next_page returns current job pending --> complete: submit_page proves page pending --> skipped: worker fails or exits without submit skipped --> pending: resume resets skipped work complete --> generating: more pending jobs complete --> finish: no pending jobs remain finish --> [*] generating --> planning: resume sees source drift and clears plan

Why a model plan must be normalized

It is tempting to regard planning as a harmless preliminary conversation and page generation as the real work. OpenWiki's design shows why that separation is misleading. A plan determines what the model is allowed to change, how later recovery identifies progress, and whether known evidence problems receive attention. If the plan stayed as unconstrained prose, two runs could interpret the same intention differently, or a planner could accidentally omit a page whose facts are already known to be stale.

Normalization supplies stable semantics without taking all judgment away from the model. The model still chooses the conceptual page boundaries and the purpose of each page. The core enforces the conditions under which those choices can safely become work: valid paths, non-conflicting deletions, deterministic ordering, required navigation, and mandatory review of known issues. It also keeps source seeds as starting points rather than treating them as an exhaustive research boundary. A page worker can therefore investigate the assigned topic while the lifecycle still has a precise ownership boundary.

The same principle explains why the host integration exposes a small fixed protocol instead of giving a coding agent direct access to OpenWiki's internal files. The host submits a plan, asks for the current job, writes the page through its own tools, and submits sparse completion decisions. OpenWiki owns the queue and the state transitions. The interface is narrow enough that the two systems can evolve independently without moving responsibility for durability into an untrusted model context.

State over time: checkpoint, resume, and source change

The run's durable memory is openwiki/.run.json. It contains more than a cursor. The checkpoint records the run identifier, mode, lifecycle phase, resolved language, source fingerprint, planning context, producer identities, prior update metadata, preparation state, and—after planning—the normalized page queue. A schema validator rejects malformed state instead of discarding it. Losing a malformed checkpoint silently would be worse than stopping, because it could turn recoverable work into an unexplained new run.

Writes use a temporary file created exclusively for that process and then rename it into place. The in-memory run is advanced only after the new state is on disk. This ordering gives a simple rule for crash recovery: disk is authoritative. If the process dies before a checkpoint write succeeds, the old durable state remains; if the write succeeds, the completed transition can be reconstructed.

The page manifest adds another kind of continuity. It records which source baseline a completed page corresponds to. On a persistent checkout, an interrupted run can resume its ordered queue, and completed pages become the evidence that earlier work need not be repeated. A page marked skipped is not considered done. Its pre-worker Markdown and Claims sidecar are restored, the run records an interrupted update, and a later resume resets that job to pending for a fresh attempt.

Resume is safe only while the source identity used by the plan remains valid. OpenWiki fingerprints model-visible repository inputs, including the relevant Git and file state, while excluding generated OpenWiki output and ignored paths. If that fingerprint changes before a resumed run continues, the old plan is deleted and the phase returns to planning. The reason is not that the prior plan was necessarily poor; it was made against a different source state.

The same honesty rule applies at the end of a run. Finish checks for source drift before and after deterministic finalization. If the repository changes during that window, OpenWiki still finalizes the wiki it can prove, records interrupted metadata, and returns sourceChanged: true. The native runner tells the operator to run another update. It does not loop automatically, because an unbounded replan could keep chasing a repository that is changing underneath it.

This is also why an update no-op requires more than a clean working tree. Claims preflight must find no stale or unresolved evidence, existing pages must have complete baseline coverage, and the source must remain stable while the no-op result and metadata are published. A no-op is consequently a small proof that the current artifact still corresponds to the current source, not merely an optimization based on elapsed time.

The queue's real purpose

OpenWiki's page queue is best understood as a boundary between flexible reasoning and durable change. The model can propose page concepts and write explanations. The lifecycle decides which page is current, which transitions are legal, what survives interruption, and whether the source version still matches the work being accepted.

That design gives the next question a precise place to land. A completed Markdown file is not enough to tell whether its factual content remains valid. To maintain a page without rewriting its entire history, OpenWiki needs a second kind of state: a record of the material propositions the page makes and the evidence that supports them.

References