CXL Makes Memory Placement a Measurement Problem
This series studies NEMO—“Nimble and Expressive Memory Observability”—and the problem it is designed to solve in CXL-based servers. The immediate question is not simply how to attach more memory, but how an operating system can tell which memory is valuable, which pages should move, and which tenant is consuming scarce bandwidth. The answer requires a mental model of telemetry: what it can see, how quickly it reports change, how precisely it attributes activity, and what the observation costs. NEMO is useful to examine because it puts that question inside the memory controller while leaving the policy decision with the OS.
When memory no longer has one price
A conventional server encourages a convenient fiction: memory is one large pool whose locations differ mainly for engineering reasons. CXL makes that fiction harder to maintain. A server may combine local DRAM with memory attached through a CXL link, and a request's latency and available bandwidth can depend substantially on where it is served. The capacity is valuable, but it is not interchangeable in performance terms.
That difference turns placement into an ongoing decision. A page that is accessed constantly can justify residence in a faster tier; a cold page may be better placed in slower, larger capacity. A workload can also change its hot set, making yesterday's good placement a source of today's slowdown. A transparent huge page creates another kind of mismatch: a full 2 MiB region may occupy scarce fast memory even when only a small part of it is active. In a multi-tenant server, the question is not only where data belongs but who is creating bandwidth pressure and how quickly the operator can respond.
The same capacity can therefore have different value at different moments. During one interval, a page may be worth moving upward because many requests depend on it. Later, retaining that page in the fast tier may crowd out a newly active region. A policy that sees only total occupancy cannot distinguish those states. Nor can a single bandwidth number tell whether pressure comes from one hot page, many cold pages, or one tenant whose activity is interfering with another. The physical hierarchy creates the possibility of better placement, but only an observation of behavior can tell the OS how to use it.
These are control problems. The operating system must observe behavior, turn observations into a policy-specific signal, and then migrate, split, or throttle something. Capacity configuration alone cannot provide that feedback. The NEMO paper describes memory as a major server cost—40–50% in its framing—so decisions about where requests are served have both performance and economic consequences. A telemetry error is not merely an inaccurate report; it can cause a policy to promote the wrong page, preserve the wrong huge page, or react after a neighboring workload has already damaged another tenant's latency.
The loop in the diagram is the central subject of this series. CXL supplies a hierarchy with meaningful differences. Linux supplies the mechanisms for exposing and managing that hierarchy. NEMO supplies one way to observe requests with enough detail for the OS to make a decision. The value of that design can be judged only by asking whether the signal arrives in time, describes the right unit, and costs less than the performance it is meant to protect.
Five questions a telemetry system must answer
The NEMO paper gives this judgment a useful vocabulary. Coverage asks what fraction of memory accesses are directly observed. A counter that sees only a sample, or only the cache misses visible at one level, may leave important traffic unknown. Timeliness asks how long it takes for a change in access behavior to appear in the signal. A complete report that arrives after a hot set has already moved is not useful for a fast control loop.
Granularity asks what the access can be attributed to: a cache line, page, region, object, or tenant. Granularity is not automatically the same as accuracy. A socket-level bandwidth number may be numerically precise while being too coarse to decide which virtual pages should move. Conversely, a page-level count may be detailed enough for placement but not enough to explain interference between tenants. Flexibility asks whether the OS can define the accesses and groupings that a new policy needs, rather than accepting semantics fixed when the hardware was designed. Overhead asks what CPU cycles, memory, interrupts, storage, and controller resources the observation consumes.
These dimensions interact. Increasing sampling can improve coverage and timeliness, but it also increases CPU work. Tracking every access at a fine granularity can produce a valuable signal, but the state needed to hold it may compete with the controller's limited SRAM. A coarse counter can be cheap and current yet unable to distinguish a tenant, a page, or a hot subregion. A flexible software routine can change its definition quickly, but if it must handle each event on a CPU, its cost may force the system to observe only a small fraction of memory.
The dimensions also describe different failure modes. Low coverage can make a policy confidently wrong because it never sees the traffic that matters. Poor timeliness can make a correct count operationally useless after an access pattern changes. Coarse granularity can turn a placement decision into a guess about which page or tenant is responsible. Low flexibility can make a counter valuable only for the workload imagined by its designer. Excessive overhead can erase the benefit of a better decision by taking cycles away from the application. The question is therefore not whether a mechanism is “hardware” or “software.” It is whether its combination of five properties matches the policy that will consume the result.
Why NEMO is the right subject for the question
Software mechanisms show why flexibility alone is not enough. Soft page faults can, in principle, capture accesses directly, but handling faults is expensive and practical coverage is limited. Page-table scans reduce the direct fault cost but see state only at scan intervals and provide much less information about intensity. Sampling mechanisms such as Intel PEBS can recover access locations more cheaply than observing everything, but their coverage and timeliness depend on a rate the CPU can afford. The paper also notes that sampling can miss traffic such as prefetches and non-temporal accesses—traffic that still reaches memory and can matter to a CXL policy.
Hardware counters show the opposite weakness. They can be timely and inexpensive because the controller or CPU already sees the traffic, but their meanings are often fixed: per-core misses, socket-level activity, or a vendor-defined bandwidth category. Such counters may reveal pressure without identifying the responsible pages or tenants. Research systems can hard-wire a particular hot-page or cache-line policy, yet a data center's next policy may need a different granularity or grouping. An efficient number is not necessarily a useful explanation of what the OS should do next.
NEMO occupies the gap between those approaches. It adds a small, off-data-path telemetry engine to memory controllers and lets the OS define policy-specific rules. The architecture can filter requests, map addresses to selected state, update counters, and optionally notify the host when a condition is met. The original memory operation continues on its normal path; the telemetry engine maintains an observation beside it. That placement aims to preserve broad coverage without asking the CPU to process every event, while the OS-facing rules keep the meaning of the signal from being frozen into one use case.
The required grouping changes with the decision. A hot-set policy may need one count for each tracked huge page. A page-splitting policy needs to see how activity is distributed inside that huge page. A noisy-neighbor policy may need many pages to contribute to one tenant counter. These are different views of the same stream of memory requests. NEMO's significance is not just that it counts quickly; it is that the controller can maintain views whose unit of meaning is chosen by the policy.
The paper tests that idea in three different decisions: HeMem's response to a changed hot set, MEMTIS's selection of transparent huge pages worth splitting, and Linux control of a memory-bandwidth noisy neighbor. Its headline results are substantial—roughly five times faster hot-set recovery, up to 10.4 times faster split-candidate detection, and about 350 times lower detection overhead in the reported scenarios. But those numbers belong to named workloads and a CXL-attached FPGA prototype, not to every CXL device or production memory controller. The important claim is narrower and more useful: when the OS can request the granularity and grouping its policy needs, a better signal can change the policy's practical response.
That is the standard by which NEMO should be read. It is not an argument that hardware telemetry makes software policy unnecessary, nor that a programmable controller can perform arbitrary analytics for free. It is an attempt to move the observation of memory behavior to the place where requests are visible, while preserving enough control for the operating system to decide what matters. The next step is to see how Linux creates the CXL tier that this feedback loop acts upon—and why exposing capacity is only the beginning.