The Page That the Guest Used but the Host Charged Elsewhere
This chapter follows memory through a microVM: from a guest file access, across a virtio interface, to the host process that makes a physical page resident. The specific focus is Linux memory cgroup charging and the two file-backed mapping patterns reported for Kata-based containers. Understanding that path lets us explain why a guest memory metric can level off while host page cache and backend usage continue to grow, without mistaking one experiment’s amplification for an unconditional production ratio.
A memory limit is meaningful only if the system agrees on what counts as the workload’s memory. In a conventional process-based container, the process that faults in a page is usually a process in the container’s cgroup. In a microVM, the guest application and the host process that supplies its file data are separated by a virtual device boundary. The guest still requests the data, but the host backend may be the process that actually performs the work that brings the corresponding page into memory.
The first touch happens on the host side
Linux memory control groups, or memcg, do not charge a page merely because an application conceptually requested it. In the model relevant to this study, a physical page is generally charged to the cgroup of the process that first brings that page into memory. This is often called the first-touch principle. It is a useful rule for a shared host because the kernel needs a concrete process and cgroup to associate with the physical page. It becomes less intuitive when one operating system asks another layer to fetch file data on its behalf.
Consider a guest program reading a file-backed dataset. At the application level, the answer to “who used the data?” is straightforward: the program in the container. The guest kernel handles the access and the virtual I/O frontend communicates with a host-side service. In a Kata configuration using virtio-fs, that service can be virtiofsd; in other arrangements the relevant process may be QEMU or the VMM. The host-side process loads or writes back file-backed data as part of serving the guest request. When the host kernel sees that process cause a page to become resident, first-touch charging can associate the page with the backend’s cgroup rather than with the guest workload’s pod cgroup.
This is not a claim that the guest has no relationship to the page. The guest requested the access, and the page becomes available in the guest’s address space. The mismatch is between causation at the workload level and ownership at the host accounting level. The guest’s use is real; the host charge is real; the selected cgroup is simply attached to the process that performed the host-side touch.
The distinction matters because the host has more than one cache and more than one view of memory. The guest kernel can report memory for the workload and can reclaim file-backed pages when it is under pressure. The host can retain corresponding data in a page cache and charge that cache to a backend process. A container metric may therefore suggest that file data has been reclaimed while the host still holds the physical data and the backend’s memory footprint continues to reflect it. The system is not necessarily violating the host kernel’s local charging rule. The problem is that the rule’s chosen owner does not match the tenant boundary the platform intended to enforce.
The paper’s tests use a differential idea: compare pages actually accessed by the guest with pages charged to the guest’s host cgroup. Pages that appear in the first set but not the second are under-accounted relative to the container. This is a more precise diagnosis than simply comparing two high-level memory counters. It asks which physical pages exist, which process cgroup owns them, and whether the workload’s access caused the gap.
The mechanism can be summarized as a lifecycle:
The diagram separates two transitions that are easy to merge in casual explanations. The guest can reclaim its file-backed view, while the host page cache remains resident for a time. And the guest can reach an out-of-memory condition even while the host still has a large cache of data that originated from the guest’s access. The result depends on cache policy, page state, memory pressure, and backend behavior; it is not a universal state machine for every file operation.
Backend choice changes the details. The paper describes Kata configurations using virtio-fs and, in confidential arrangements, virtio-9p, where host-side filesystem services handle guest file operations. Firecracker commonly uses virtio-blk, in which the guest sees a virtual block device and the VMM handles the corresponding requests. These designs all cross a guest/host boundary, but they expose different objects and processes. A finding about a virtiofsd-served file-backed page should not be silently generalized to every block-device configuration.
Two mappings, two failure shapes
The study reports two memory-accounting patterns that both exploit the gap between guest access and host charging, but they are not interchangeable. The first uses writable shared file-backed mappings. The second uses private read-only file-backed mappings. The mapping flags describe how processes and files relate to the pages, so changing them can alter program semantics even when the accounting result looks similar.
Writable shared file-backed pages
A shared mapping gives writes a file-backed destination that can be visible through the mapping’s shared relationship. In the reported Kata experiment, a workload writes to file-backed pages through a shared mapping. The host backend is involved in loading or managing the file data, and the physical pages are predominantly charged to the host-side overhead process rather than to the pod. The study’s table records only a small newly charged amount for the pod compared with the much larger amount associated with the backend in the test.
The important point is not that a particular mapping flag magically disables memory accounting. The host kernel still charges the page somewhere. Rather, the operation makes the backend the process that first populates the relevant physical page, so the charge follows the backend’s identity. If that identity is outside the workload’s enforced budget, the guest can appear to use less memory than the host is retaining on its behalf.
The shared case also carries an obvious correctness implication. A shared mapping is not equivalent to an ordinary private allocation. Writes may be visible through the file or to another process that maps the same object. A program that expects private state can behave differently if its allocation is redirected into shared file-backed storage. The accounting path and the program’s data-sharing semantics are linked.
Private read-only file-backed pages
The second pattern uses a private mapping and read operations. A private mapping normally provides copy-on-write behavior if a process later writes, while a read-only access does not itself modify the mapped file. The study reports that reading a private file-backed mapping can still produce little new charge in the container because the host backend loads the pages on behalf of the guest. The memory is physically useful to the workload, but the host process remains the first toucher for the page population observed in the experiment.
This case is distinct from the shared writable case in both behavior and risk. A read-only access can preserve the application’s expectation that it is not modifying the underlying file, and private mapping semantics matter if a later write occurs. But “read-only” does not mean “free” in physical memory. The pages must still be loaded, retained, and eventually reclaimed somewhere. The issue is who receives the host charge while that happens.
The distinction is important for evidence as well. A test that writes shared pages does not by itself prove that reads from private pages behave the same way. The authors tested both patterns and treated them as separate attack vectors. That is stronger than inferring one from the other, but it remains bounded by the tested page types, backend, cache policy, and cgroup configuration. File-backed mappings in another microVM or with another host integration may follow a different path.
Reuse after a container is destroyed
The study also examines what happens when two containers in a pod use the same file-backed pages over time. After container A is destroyed, container B accesses the same 128 MiB of file-backed pages. The reported result is that B incurs negligible newly charged memory compared with the original population. In the tested setup, the resident file pages and their host-side accounting history are not immediately equivalent to a fresh allocation charged to B.
This observation matters because accounting is a lifecycle, not a single event. A page can be created, shared through a file-backed cache, reclaimed from one view, reused by another container, and eventually reclaimed by the host. Destroying a container removes its processes, but it does not necessarily make every physical page that those processes touched disappear at that instant. The result should be stated as a tested lifecycle behavior, not as a universal rule that memcg never reassigns file-backed pages.
Pressure reveals what the limit actually governs
The most striking local result is easy to misread. The paper describes a Kata container with a 64 MiB memory limit accessing roughly 32 GiB of mapped pages—about 500 times the configured limit, with the paper’s presentation using a nearby rounded amplification. That number is not a claim that 32 GiB of arbitrary anonymous memory can always be placed beyond a 64 MiB limit. It describes a file-backed workload, a particular backend and cache policy, and a measurement of the difference between guest-visible accounting and host-resident data.
The guest still experiences pressure. File-backed pages are treated as reclaimable candidates, so the guest can evict them as it tries to stay within its memory budget. But reclaiming a file page from the guest’s active cache does not necessarily erase the corresponding host-side page cache immediately. The host may retain the data, and a later access may fault it back in through the same delegated path. The workload can therefore exchange memory pressure for additional I/O and page faults rather than simply holding every page as anonymous resident memory.
There is another limit that the headline ratio hides: address-space bookkeeping. In the reported experiment, accessing a very large mapped range causes the guest’s page-table usage to grow. The paper observes that a 32 GiB virtual address range can consume enough page-table memory to hit a 64 MiB guest limit even though the file-backed data itself is being reclaimed. This is why the guest may eventually trigger an out-of-memory kill while host page cache remains large. Virtual address space, page tables, guest file cache, host page cache, and physical capacity are different resources with different accounting paths.
That behavior provides a useful diagnostic pattern. If guest memory.current rises to its limit and then remains near it, while guest file usage is reclaimed and host page cache grows with the workload, the numbers are telling a layered story. The guest is enforcing pressure on the resources it can reclaim or charge. The host is retaining file-backed data and charging the process that populated it. A low guest number is not proof that the host is idle, and a high host number is not proof that the guest has bypassed every limit; the two metrics refer to different states.
The practical boundary is therefore narrower and more precise than “memory accounting is broken.” The reported behavior requires a delegated file-access path, a page type and mapping pattern that follows that path, a host observer that charges the backend rather than the pod, and enough host capacity for the retained data to matter. Change the backend, mapping, cache policy, cgroup placement, or available memory and the outcome may change. What remains general is the method of analysis: find the physical page, identify the first toucher, and compare that owner with the resource boundary the platform intends to enforce.
With the page-level mechanism in view, the next problem is to combine it with the other side of the resource boundary. Memory asks which process owns a physical page. Storage asks which path or object the accounting system can see. The same guest/host split can make those questions diverge, but the lifecycle of a file requires a different kind of tracing.