When the Cost Is Not in Your Code: Kernels, Allocators, and Workload Shape

An ARM MTE slowdown does not necessarily belong to the application that appears slow. The focus here is the full path around a tagged workload: kernel policy, allocator work, memory mapping, and the boundary between local and remote measurement. Following that path makes two practical judgments possible: whether a regression is introduced outside the application’s hot loop, and whether a targeted fix is better than changing MTE mode or disabling the protection.

The Memcached regression begins as the wrong kind of symptom

The USENIX study evaluates MTE on server workloads including RocksDB, Nginx, PostgreSQL, and Memcached. RocksDB, Nginx, and PostgreSQL generally show overheads below the largest SPEC CPU outliers. Memcached looks different in the initial colocated tests: on many cores, some configurations slow by as much as 1.40×, with the largest effects appearing when the ratio of reads to writes is high and the workload runs across many cores.

A first explanation might blame the application’s own memory traffic. Memcached is an in-memory key-value service, so a slowdown that grows with request load and core count appears, at first, to be a direct cost of checking its data. But the study finds that the regression persists even when the application does not use MTE itself and MTE is enabled only at the kernel level. That observation changes the location of the investigation. The question is no longer only how Memcached touches its heap. It is what the kernel and processor do when MTE is present in the surrounding system.

The distinction matters for diagnosis. If an application’s own tagged accesses are the cause, a team might examine allocator policy, access locality, or the mode. If kernel checks on untagged kernel memory are the cause, changing application code cannot remove the work. A performance number can therefore be accurate and still point to the wrong remedy if the benchmark boundary is treated as the ownership boundary.

Why kernel policy can charge the application

The kernel needs to preserve and manage tag metadata for user pages that it maps into its address space. It also needs a policy for whether accesses to user and kernel mappings perform tag checks and whether a mismatch raises a fault. The study describes an ambiguity in the relevant Arm behavior: a configuration that suppresses tag faults does not necessarily disable the tag-check work itself.

That distinction between checking and faulting is the key. A kernel can arrange not to terminate on a mismatch while still allowing the processor to perform the comparison. If the kernel’s own accesses target memory that is considered taggable, those comparisons can consume hardware resources even when no user-space tag fault will be delivered. On the tested AmpereOne system, tag checks continued for untagged kernel memory. The tested Pixel systems avoided the same overhead by disabling the checks rather than merely suppressing their faults.

This is a hidden coupling between an application and the operating system. Memcached does not need to call an MTE instruction for its requests to be affected. The request path enters the kernel, and the kernel’s mapping and check policy determines whether additional metadata work is done around those transitions.