Inside NEMO: A Programmable Pipeline at the Memory Controller

NEMO's central mechanism is a small, bounded pipeline beside the memory data path. The question here is how one request becomes a policy-specific counter or alert without making the CPU handle every event or delaying the original memory operation. Following that path—from request filter to translation, update, readout, and notification—also explains why NEMO can represent page, sub-page, and tenant views while stopping short of arbitrary analytics.

A request becomes a bounded state update

A memory controller already receives the metadata needed to serve a request. NEMO taps the request header and sends that metadata to one or more telemetry pipelines while the original operation continues toward DRAM or CXL memory. The pipeline is an observer, not an extra destination for the application's data. It does not fetch the requested cache line, rewrite the address, or wait for the operation to complete before the normal memory path can proceed.

Each pipeline is configured for one telemetry. A controller can therefore host separate views for different policies, and each pipeline can snoop the controller's memory channels. The design places the observation where traffic is visible at high rate, but keeps the policy meaning in configuration and driver state. That is the first important boundary: NEMO observes request metadata, while the ordinary controller logic remains responsible for serving the request.

flowchart LR A["Memory request header"] --> F["Filter: read/write and address range"] A --> M["Original request to DRAM or CXL"] F --> K["Match: derive a mask/shift key"] K --> X["Translation: base index plus offset"] X --> U["Update: banked SRAM state"] U --> Q{"Predicate true?"} Q -->|No| R["Driver reads state"] Q -->|Yes| I["Optional host notification"] R --> D["Aggregate or reset state"] I --> D

The filter is the first policy choice. A telemetry can accept reads, writes, or both, and can be limited to an address range. Filtering prevents one pipeline from spending state on traffic outside the question it was installed to answer. For a page-hotness view, the relevant range may be the selected huge pages. For a tenant view, the filter can cover the memory served by the controller while the translation table determines which addresses belong to which tenant state.

The match stage turns a physical address into a lookup key. NEMO uses fixed-latency arithmetic based on masks and shifts rather than an unbounded search. The mask selects relevant address bits, and the shift establishes the primary region size. A key can represent a 2 MiB huge page, a much larger aligned region, or another power-of-two unit supported by the configuration. If a key is absent or invalid, the request does not update that telemetry state.

The update stage receives the selected state index and, optionally, part of the request as an operand. It performs a read-modify-write using the configured operator. The simplest example adds one to a counter for every matching request. Other supported operators can update state with different fixed operations, but the important property is that the work is bounded and predictable. A telemetry rule is not a callback that can run arbitrary code for every access.