Overview

M3U treats pass-through I/O faults as a device-queue problem: DMA reaches non-present memory through PCIe and host interrupt handling, so one missing descriptor can stall many requests. It drains VirtIO work, parses active rings and descriptors, and pre-installs the small predictable state before the guest and device resume. In the tested configurations, IOPFs fell by up to 98.5%, but a buffer-allocation race remains and capture, compatibility, and cost stay device-specific.

Learning outcomes

IOPFs can block a device queue, making them more disruptive than an ordinary vCPU fault.
Pre-installing active VirtIO descriptors trades 94–304 milliseconds of measured device-state migration work for fewer early IOPFs.
M3U reduced tested IOPFs by up to 98.5%, but guest queue timing can still create residual faults.

Keywords

M3UVirtIOIOPFDMADescriptorsQueues

The Third Path: Make Device Faults Unnecessary

M3U treats pass-through I/O as a separate post-copy problem because a device fault is not just a slower version of a vCPU fault. A device can issue DMA to memory that the destination has deliberately marked non-present, and the resulting I/O page fault can stall an entire queue. This chapter follows that path and explains why M3U pre-installs a small, predictable set of device state before the guest resumes, then measures the benefit without claiming that every device fault disappears.

Why a device fault is not an ordinary page fault

A vCPU page fault begins with guest execution. The guest instruction touches an address, the destination discovers that the corresponding page is missing, and the user-space migration path asks the source for the contents. The affected guest thread may wait, but other guest work can sometimes continue. A pass-through device follows a different path. Its DMA engine can access guest memory without first executing a host CPU instruction that the ordinary page-fault path can handle.

When that DMA reaches a non-present page, the device generates an I/O page fault, or IOPF. The request travels through a PCIe transaction and a host interrupt before software can service it. The fault is therefore inserted into the device's own critical path. A network controller or storage device may have to stop processing the affected I/O queue until the missing page is supplied. Under bursty traffic, the queueing consequence can be larger than the time needed to fetch one page: packets or requests wait behind the unresolved DMA, and the guest service sees a delay that can spread beyond the first access.

The ownership difference is important. A vCPU fault is observed by the host while software is already handling a guest execution path. The migration process can associate the fault with a guest address and arrange for that thread to sleep. An IOPF is raised by a device that expects the memory transaction to complete as part of its own queue protocol. The host must service the fault without losing the device's position in that protocol. That extra boundary is why a device can turn one missing page into backpressure for many subsequent operations.

The M3U paper compares this path with vCPU demand paging in its evaluated hardware setup. It reports roughly twice the average IOPF latency and up to 3.21 times the maximum latency in the comparison. Those figures belong to the tested platform and hardware support; they are not a universal ratio for every pass-through device. They do establish why the system should avoid putting a device on the remote-page critical path when it can identify state that will be needed immediately after resumption.

Timing is the first clue. The paper finds that almost all IOPFs in its post-copy experiments occur within the first few seconds after the VM resumes. That concentration is different from a general memory workload in which page faults appear wherever the guest's working set happens to go. Device resumption creates a predictable burst: the device begins processing queues whose metadata and active descriptors have not necessarily arrived at the destination yet.

The structure of the I/O state is the second clue. M3U distinguishes two broad kinds of pages in the VirtIO path.

Published by DeepMox

FAQ

Why can an IOPF stall more than one guest thread?

An IOPF can stall more than one guest thread because it enters a pass-through device’s I/O queue, which may stop processing until the missing DMA page is supplied. Packets or requests can accumulate behind that unresolved transaction.

What device state does M3U pre-install?

M3U pre-installs active VirtIO ring structures, descriptor tables, and referenced buffers identified from pending or in-flight queue state. It captures the predictable device path rather than copying every page near a queue.

Why does M3U not eliminate every IOPF?

M3U does not eliminate every IOPF because a race can occur between guest buffer allocation and posting that buffer into a descriptor. The design preserves an unmodified guest driver, so it accepts this residual timing window.

Previous10 minM3U: Lock Relaxation for Post-Copy

Next12 minM3U: Migration Results and Limits

M3U: Pre-Installing VirtIO State

The Third Path: Make Device Faults Unnecessary

Why a device fault is not an ordinary page fault

FAQ