Reading the Frontier: Model Sizes, NAS, and Benchmark Conditions

For RF-DETR, a benchmark table is not a list of universal properties; it is a measurement of a particular artifact under a particular protocol. The useful question is how to read the reported frontier without collapsing accuracy, latency, throughput, domain shift, and availability into one score. That distinction lets the reader compare RF-DETR variants and competing real-time models fairly, then identify which measurements still have to be collected on their own data and hardware.

The architecture explains why the family can offer several points on a curve, but it does not decide which point a deployment needs. A small model at a tight latency budget and a larger model with more spatial detail are solving different operational problems. The benchmark’s job is to make those differences visible; the reader’s job is to preserve the conditions while interpreting them.

AP is not one number

The primary detection metric in RF-DETR’s tables is COCO AP50:95. It averages precision over intersection-over-union thresholds from 0.50 to 0.95 in steps of 0.05. A prediction must therefore remain useful as the required overlap becomes stricter. AP50 is easier to satisfy because it evaluates only the 0.50 threshold. A model can look strong at AP50 while losing more of its score at stricter overlaps if its boxes are less precisely localized.

That distinction matters because different application errors have different costs. A rough box may be sufficient for a coarse people counter, while a robotic grasping system or an inspection workflow may need tighter localization. Reading only the AP50 column can hide that difference. Reading only AP50:95 can also hide whether a model’s advantage comes from broad detection recall or precise boundaries. The metric is a lens on the error, not a complete description of application quality.

Keypoint detection adds another distinction. Its COCO score uses object-keypoint similarity, or OKS, rather than box IoU. The keypoint preview’s AP50:95 is therefore not directly interchangeable with a detection model’s AP50:95, even though the notation looks similar. Segmentation has its own mask metric as well. A family can use related evaluation language across tasks while measuring different geometric objects.

Latency has an equally important definition. RF-DETR’s documented latency numbers are single-image inference latency on an NVIDIA T4 using TensorRT, FP16 precision, and batch size one. The benchmark documentation specifies TensorRT 10.4 and CUDA 12.4, and inserts a 200-millisecond buffer between passes to reduce variation caused by power and thermal effects. This is a reproducibility choice, not a maximum-throughput measurement. A service processing a continuous batch, a webcam loop using Python preprocessing, and a mobile runtime may have very different bottlenecks.

Parameter count is also defined operationally. The tables use deployment, fused nn.Module parameter counts rather than the raw tensor count in a saved training checkpoint. That makes the number more relevant to the deployed artifact, but it still does not tell the reader how much memory the runtime, activations, preprocessing, or postprocessing will consume. Model size is a useful proxy for capacity and cost; it is not a full resource profile.

The evaluation side is similarly bounded. The project says the COCO rows were re-evaluated in-house with pycocotools over the full 5,000-image val2017 split, so the listed rows share a comparison protocol.