Why Existing Memory Telemetry Misses the Policy Signal
The central problem after a CXL tier is visible to Linux is not a lack of measurements; it is a mismatch between what a mechanism observes and what a policy must decide. This chapter compares software observation, conventional hardware counters, and fixed-function research telemetry using coverage, timeliness, granularity, flexibility, and overhead. The goal is to see why NEMO targets a middle position: controller-side observation that can keep up with traffic while allowing the operating system to choose the grouping and meaning of the signal.
Software buys flexibility with work and delay
Software telemetry is attractive because software can change its interpretation without changing the memory controller. If a policy wants a different page set, sampling rule, or aggregation, the operating system can often alter the code or the interval. That flexibility is important in tiered memory, where one workload may need hot-page tracking and another may need tenant-level bandwidth attribution. But the observation still has to be triggered and processed somewhere, and the CPU is usually the first place that cost appears.
Soft page faults illustrate the strongest version of the software approach. The operating system can deliberately make a page unavailable or read-only and infer an access when the application faults. In principle, this can provide very direct coverage. In practice, handling faults consumes CPU time, so systems can apply the method only to a limited subset of memory. The page itself also remains the unit of observation: the mechanism does not automatically reveal how activity is distributed inside a large page or which tenant caused each access.
Page-table scanning avoids the direct cost of handling a fault on every observed access. The OS periodically walks page tables and inspects accessed and dirty bits, producing per-page read and write information. The price is timeliness and intensity. A scan sees a state accumulated since the previous scan, not a continuous account of every event. Each page effectively supplies a small signal for an interval, so the policy cannot tell whether one access or a burst of accesses produced the same bit. The paper gives a production example in which scans run every 30 seconds. That may be reasonable for a slow-moving decision, but it is a poor fit for a hot set that changes faster than the scan period.
The trade-off is not solved by scanning more often. A shorter interval improves timeliness, but it requires more page-table work and more frequent processing. A larger region or coarser page size reduces work, but it weakens granularity. The system must choose whether to know less about more memory or more about a smaller region. That is a genuine coverage-versus-overhead decision, not merely a tuning detail.
Sampling cache misses with Intel Processor Event-Based Sampling, or PEBS, moves to another point in the design space. Instead of processing every event, the system processes a selected fraction and reconstructs access patterns from sampled locations. Sampling can provide useful attribution with less CPU work than fault-based observation, but its coverage and timeliness depend on the sample rate and collection frequency. Increasing either improves the chance of seeing a change while increasing processing cost and interrupt pressure. The paper cites prior work in which sampling 10% of cache misses caused more than 50% application slowdown, illustrating why production systems normally choose much lower rates.
Sampling also changes what “an access” means. PEBS-based observation is tied to the events exposed by the performance-monitoring unit, such as cache misses, rather than every request that eventually reaches a memory controller. The NEMO paper specifically notes that this can miss prefetches and non-temporal accesses. Those operations may not produce the same sample as an ordinary demand load, but they still consume memory bandwidth and can affect placement or interference. A policy that uses samples as a proxy for controller traffic may therefore be precise about the events it captured while remaining incomplete about the resource it is trying to manage.
The virtualization boundary adds another limitation in the paper's comparison: PEBS cannot profile VM guests from the host in the cited setting. That matters to cloud operators because the host may need to attribute physical-memory behavior without asking each guest to reveal its own activity. The point is not that PEBS is useless. It can remain a practical component for a tier or event that the CPU can observe cheaply. The point is that flexibility at the software layer does not guarantee full coverage of the traffic that a CXL policy cares about.
Consider a hot-set change as a simple stress test. A fault-based method may observe the new accesses directly, but the fault handler competes with the application. A page scan may preserve low per-event cost, but the new hot pages can remain invisible until the next scan. Sampling can shorten that delay by increasing its rate, but the CPU budget places a ceiling on how far the rate can rise. None of these is intrinsically wrong; each makes a different promise about which part of the control loop it can afford to keep current.
The same comparison changes when the policy asks about the slow CXL tier rather than the CPU's cache. A sample that identifies a cache miss may be useful evidence, but it is still a proxy for traffic at the controller. Prefetches and non-temporal operations can consume the tier's bandwidth without appearing in the sampled event stream. The policy then faces an attribution problem: it may know exactly where its samples came from while undercounting the activity that determines latency and interference.
Hardware buys efficiency with fixed meaning
Hardware counters invert the trade-off. A memory controller already sees requests as it serves them, so a counter can increment without a software interrupt for every event. CPU and memory-controller counters can be timely, high-coverage, and inexpensive to read relative to fault handling or high-rate sampling. That makes them attractive for continuous monitoring.
Their weakness is attribution. A per-core LLC-miss count says something about pressure at a core; a socket-level memory-controller count says something about aggregate activity. Neither necessarily identifies the virtual page, object, or tenant responsible for the pressure. Such a counter can tell an operator that a tier is busy while leaving the policy to guess which pages should migrate or which tenant should be throttled. Intel Memory Bandwidth Monitoring can attribute bandwidth to processes, but the paper describes accuracy issues and the absence of options such as filtering reads from writes.
Fixed semantics also age poorly as memory policies change. A hardware design can be excellent at tracking a top set of hot pages or a selected cache-line property, but it has to decide its granularity, coverage, and aggregation rules before the hardware is built. A later policy may need a different page size, a tenant grouping, or a read/write distinction. If the required view is not present, the operating system falls back to software aggregation, giving back some of the efficiency that motivated the counter.
The problem is especially clear when the unit of measurement and the unit of action differ. A socket counter can establish that a memory controller is busy, but a placement policy acts on pages. A per-core miss counter can identify a busy execution context, but a cloud isolation policy may need to charge a tenant whose work spans many cores. A process-level bandwidth counter may be closer to the policy's unit, yet still lack a way to filter the traffic or distinguish the read/write behavior that matters to a particular control rule. The counter is doing exactly what it was designed to do; the policy is asking a different question.
Fixed-function research telemetry narrows that mismatch for a chosen task, but the same benefit creates a new boundary. If the hardware permanently defines “hot” as a particular ranking or tracks only a predetermined number of regions, it can be efficient for that policy and irrelevant to another. A CXL deployment can change its tier ratios, tenants, or page-management strategy without changing the physical memory controller. Flexibility therefore has a lifecycle value: it protects the observation layer from becoming obsolete when the policy layer evolves.
This is why a comparison based only on “hardware versus software” is misleading. Software is not one option and hardware is not another with a single score. Each mechanism occupies a different point across the five dimensions:
- Faults offer direct observation but can spend too much CPU to cover much memory.
- Page scans reduce event cost but can be stale and weak on access intensity.
- Sampling offers useful locations at a controlled rate but misses unsampled traffic and makes timeliness expensive.
- Fixed counters offer efficient, current aggregates but may not identify the unit a policy must act on.
The right question is which failure a policy can tolerate. A slow, coarse trend may be enough for capacity planning. A hot-set migration loop needs coverage and timeliness at page-relevant granularity. Huge-page splitting needs to see distribution inside a page, not just the page's total count. Noisy-neighbor control needs attribution to a tenant and a reaction path that does not consume a large fraction of a CPU core. There is no universal winner because these decisions ask different questions of the same memory traffic.
The requirements that emerge
The comparison produces a concrete set of requirements for NEMO rather than a vague preference for programmable hardware. First, the observation point must be close enough to the served request to avoid losing important traffic between the CPU's event model and the memory controller's actual work. Second, the per-request operation must be bounded: a telemetry engine that sometimes performs an unbounded search cannot keep up with a controller's rate. Third, the state must be close to the controller so updates do not create extra memory traffic that competes with the application.
Fourth, the operating system must be able to define the unit of meaning. The same stream should support one counter per selected huge page, multiple counters inside a huge page, or one counter for all pages attributed to a tenant. That does not require arbitrary software on every request; it requires a compact translation from request metadata to a policy-chosen state. Fifth, the host must be able to read a useful aggregate without turning telemetry retrieval into another CPU bottleneck. Notifications can matter when polling would react too slowly, while reset-on-read or similar effects can make each read represent a clear interval.
Those requirements already describe NEMO's design direction, but they do not make it a universal solution. The basic pipeline is intentionally restricted to a bounded, single-step update over associative and commutative state. That restriction is what permits predictable line-rate behavior and aggregation across controllers. It also means that a policy needing multi-step arithmetic, several state locations per request, or exact history cannot simply be declared “programmable” and assumed to fit. The paper names variance and exact top-K tracking as examples that require more than the basic primitive; chained pipelines are proposed for some richer bounded sketches, not demonstrated as unlimited analytics.
The resulting judgment is conditional. Software observation remains a reasonable choice when the region is small, the policy is slow, or the required semantics change too frequently for hardware state. Fixed counters remain valuable when a coarse aggregate answers the operational question and low overhead matters more than attribution. NEMO is aimed at the middle case: a policy needs continuous, fine-grained or tenant-specific evidence, but the CPU cost of producing that evidence would undermine the policy.
That is the policy signal existing approaches often miss. It is not simply a more accurate number. It is a number grouped at the unit the OS can act on, produced soon enough to influence the next decision, and obtained without moving every observation through the host. The next step is to test whether NEMO's specific controller-side abstraction can connect those requirements into one working feedback loop rather than just another list of desirable properties.