The Boundaries: Reliability, Isolation, and Running the System
OpenViking’s boundary model is the focus here: what happens when an agent is searching context while that context is being updated, permissioned, replicated, or sent through external model services. The practical goal is to identify the authoritative data, match each failure defense to the problem it solves, and separate a convenient local demo from a service that can safely handle remote or multi-user traffic.
When a search result could be wrong
Imagine that an agent has retrieved a runbook from a project directory. While the agent is preparing its answer, someone moves the runbook, updates it, or deletes the directory. The system now has two possible failures. It can return an old index record whose URI no longer points to content, or it can preserve the source while temporarily failing to find it. OpenViking’s storage design chooses the second failure as the safer one.
The reason is an explicit asymmetry: the filesystem layer is the source of truth, while the vector index is derived state. AGFS—now implemented through the project’s Rust-based RAGFS—stores L0, L1, L2, and multimedia content. The vector index stores URIs, vectors, summaries, and metadata used to discover that content, but it does not own the file itself. If an index is lost, it can be rebuilt from the content layer. If source content is lost, a later index rebuild cannot recover it.
That principle shapes operation order. For a delete, OpenViking removes the relevant vector records first and then deletes the source file. During a failure, the file may remain longer than intended, but it should not remain visible through search as an object that no longer exists. For a move, the system copies the source to the new location before updating vector URIs and removing the old source. If index migration fails, the original source and old index can remain consistent while the copied destination is cleaned up. The design favors temporary incompleteness over a confident pointer to bad data.
This is a more useful way to think about “consistency” than assuming that every layer changes at one instant. The content store, derived index, semantic summaries, and backup backends have different jobs and may have different completion times. The system is consistent enough for a particular operation when those differences cannot cause a caller to be directed to invalid content, and when a failed stage can be retried safely.
Locks and queues solve different failures
The first defense is mutual exclusion. OpenViking uses path locks so that concurrent writes touching the same path do not interleave unpredictably. An exact lock protects a concrete file or path. A tree lock protects a directory subtree for operations such as deleting a directory or holding a resource stable while it is being processed. Move operations protect both source and destination according to whether the source is a file or directory.
These locks are not a general undo system. They do not roll back every change made inside a block. Their job is to ensure that one operation has exclusive access while it performs its source and index updates. Fencing tokens help distinguish a current lock holder from an older process that left a lock behind. Stale locks expire and can be cleaned up, so a crashed process does not permanently freeze a path.
The second defense is persistence for work that must survive a process restart. A session commit has a synchronous archival phase and a background extraction phase. If the process crashes during memory extraction, the persistent session_commit queue can resume the second phase from the archive. Semantic and embedding queue operations are designed to be idempotent and retriable, so repeating work does not require pretending that the first attempt never happened.
The distinction is important. A lock answers, “Who may operate on this path right now?” A persistent queue answers, “What work still needs to be completed after this process stops?” A lock without a durable queue can prevent live races but lose pending work. A queue without a lock can resume the same job while another operation is changing its target.