Why Zero Mean Error Is Not Agreement
For cuffless blood-pressure systems, the phrase “small error” can refer to several different quantities. This chapter examines the signed mean difference between an estimate and a reference, then separates it from individual disagreement, variability, coverage, and baseline performance. The goal is practical: to read a pooled accuracy claim without mistaking a favorable average for evidence that a wearable will be dependable for a particular person or decision.
Two reviews, two questions about accuracy
The earlier systematic review compared wearable readings with cuff sphygmomanometers in 21 prospective studies involving 897 participants. Its pooled systolic BP mean difference was 0.58 mmHg, with a 95% confidence interval from −0.35 to 1.51 mmHg and I² of 0%. For diastolic BP, the pooled mean difference was −0.33 mmHg, with a 95% confidence interval from −1.50 to 0.85 mmHg and I² of 56%. In those selected comparisons, the average wearable and cuff readings did not differ significantly.
That result answers a bounded question: did the two measurement types show a large average offset in the included studies, which generally measured participants while sitting or resting? It does not say how close every paired reading was, whether the same relationship held during activity, or whether the studies represented people who would use the device in practice. The authors themselves described the underlying evidence as limited by small, prospective, non-randomized studies and the absence of settled standards for cuffless diagnosis.
The newer review assembled a different body of evidence. It included 86 original cuffless BP studies and pooled 46 effects reporting systolic mean error. The pooled error was 1.12 mmHg, a small positive value indicating slight average overestimation, but the heterogeneity was 98.8%. The review therefore treats the estimate as a way to quantify inconsistency across studies, not as the expected error for a patient or the benchmark for one clinical use case.
The distinction matters because a pooled estimate combines results generated by different sensors, signals, models, reference standards, populations, calibration procedures, and validation splits. The earlier review's average comparability and the newer review's warning about generalizability can both be credible within their respective scopes. The first emphasizes paired average differences under selected conditions. The second asks whether the field's reported results are sufficiently consistent and independently tested to support broader claims.
What a signed mean preserves—and what it erases
For an individual observation, signed error is the estimated pressure minus the reference pressure. A positive value means overestimation; a negative value means underestimation. Averaging signed errors preserves systematic direction. If a device is consistently high, the mean can reveal that shift. But averaging also allows errors in opposite directions to offset one another.
Imagine ten paired readings in which five estimates are 6 mmHg above the reference and five are 6 mmHg below it. The signed mean error is 0 mmHg. That number correctly says there is no net directional shift in this illustrative set. It does not say that the estimates are close: every estimate is 6 mmHg away. If the errors were +15 and −15 mmHg in equal numbers, the same mean would coexist with much poorer agreement.
The issue is not limited to deliberately balanced examples. A model may overestimate at one pressure range and underestimate at another. It may perform well when a person is seated and poorly when the person stands. It may be accurate for one subgroup and systematically off for another. A single average can cancel these patterns when they are pooled. Once the observations are separated by pressure, subject, posture, activity, or time since calibration, the hidden structure may become visible.
The newer review reports a wide range of study-level results: systolic mean error values ran from −3.38 to 15.42 mmHg, while reported mean absolute differences ran from 0.018 to 11.3 mmHg. These ranges do not mean that one device necessarily produced every value, nor do they provide a direct ranking. They show that the studies did not converge on one stable level of performance. The pooled signed mean is a center of a heterogeneous literature, not a description of the tails or of any individual user's experience.
This is why “no significant difference” should not be treated as equivalent to “the devices are interchangeable.” Statistical significance tests whether the data provide evidence of a nonzero average difference under a particular model and sample. Clinical agreement asks whether discrepancies are small enough, predictable enough, and stable enough for the decision at hand. Those questions overlap, but neither one substitutes for the other.
A metric panel is more informative than a headline number
Mean deviation with its standard deviation, often written MD ± SD, keeps both average direction and the dispersion around that average. It can show that a device has little systematic bias but a wide spread. It can also be influenced by outliers, so the reader should know whether a few extreme observations dominate the summary.
Mean absolute difference, or MAD, removes the sign before averaging. It therefore answers a different question: how far away from the reference are estimates on average, regardless of whether they are high or low? MAD is often more directly connected to typical magnitude of disagreement, but it still does not describe the worst case, the shape of the distribution, or which conditions generate the largest errors. A low average absolute error can coexist with a clinically important subgroup or tail.
Cumulative percentages within 5, 10, and 15 mmHg—often written CP5, CP10, and CP15—show how much of the data falls inside progressively wider error bands. These percentages make coverage visible, but they can be read alongside the direction of error: a systematic shift can still leave a substantial fraction of readings inside a broad band. Mean absolute percentage difference, or MAPD, expresses error relative to BP level and can help contextualize why absolute discrepancies may grow at higher pressures. Correlation measures how two quantities move together, but a device can track ranking or direction while remaining consistently offset, so correlation alone does not establish interchangeability.
Bland–Altman analysis provides another perspective by plotting the difference between cuffless and reference values against their mean. It can reveal systematic bias, pressure-dependent error, and limits of agreement. The limits are important because they describe where individual differences may lie, not only where their average lies. They should be accompanied by uncertainty around the limits and interpreted with appropriate distributional assumptions. The newer review notes that none of the studies it examined reported checking whether the differences were approximately normally distributed, an omission that weakens confidence in the analysis as presented.
The review recommends reporting MD ± SD, MAD, CP5, CP10, CP15, MAPD, and correlation together with scatter and Bland–Altman plots. None of the reviewed studies supplied all of those elements at once. This is not a request for statistics as decoration. Each measure exposes a different way a cuffless BP system can succeed or fail. A single metric cannot simultaneously describe bias, typical magnitude, tails, proportionality, condition-specific behavior, and uncertainty.
Heterogeneity is evidence about the evidence
An I² of 98.8% means that almost all of the observed variation in the pooled systolic mean-error results is attributed, in the review's model, to differences between studies rather than ordinary sampling noise within one common effect. It does not identify one cause by itself. The differences may reflect validation design, sensor modality, cohort, reference device, calibration, preprocessing, pressure range, or other study features. But it makes a universal interpretation of the pooled 1.12 mmHg value implausible.
Sensitivity analyses reinforce the need for caution. A restricted-maximum-likelihood model with a Hartung–Knapp adjustment produced a pooled estimate of 0.86 mmHg with a confidence interval extending from −0.08 to 1.79 mmHg. Trimming extreme mean-error values gave 0.57 mmHg, winsorizing at a 5 mmHg cap gave 0.79 mmHg, and a bias-adjusted estimate was 0.76 mmHg. The estimates remain small and generally positive, but their magnitude and uncertainty change when the treatment of heterogeneity and extreme values changes. That stability of direction does not turn the result into a clinical threshold.
The review found no strong evidence of classical publication bias: the Egger regression was borderline rather than statistically significant, and its other diagnostics did not detect a small-study effect. Its interpretation is that the main problem is less likely to be selective publication alone than the prevalence of small, underpowered, and methodologically heterogeneous studies. A literature can therefore be broadly honest and still produce an unreliable pooled benchmark because the experiments do not test the same practical capability.
The missing baseline can make prediction look like measurement
A model needs a counterfactual. If the model uses physiological signals to predict BP, how well would a simpler predictor perform on the same cohort and split? In a population with a narrow BP distribution, always predicting the population average may yield a respectable error. A calibrated system may appear to work because its estimate stays close to a person's calibration value, even if it adds little information about subsequent change. Without those baselines, the contribution of the signal and algorithm is difficult to identify.
The newer review recommends population-average or demographic baselines for calibration-free systems and calibration-value baselines for calibration-based systems. Yet none of its 86 reviewed studies reported baseline model results; the authors found only one study that explicitly incorporated such comparisons. This omission does not prove that the models were merely guessing the average. It does mean that the reports often do not show how much predictive value came from the signal rather than from the cohort's prior distribution or the calibration step.
The review's Aurora example makes the issue concrete. In its structured model-card illustration, a multimodal model had a static MAD of 8.08 mmHg and a dynamic MAD of 8.61 mmHg. The corresponding baseline values were 10.90 mmHg in static testing and 14.48 mmHg in dynamic testing. The example suggests that the added signals can matter, particularly under the harder condition, but it also shows why the comparison must be reported. The values are a worked example, not a universal benchmark for cuffless devices.
What the result can responsibly say
A small signed mean can support a limited statement: under the tested conditions, the estimates may have little average directional bias relative to the chosen reference. To support a stronger statement, the report must show how errors are distributed, whether performance is stable across relevant pressure ranges and subgroups, how many observations and independent subjects were tested, and whether a simple baseline performs nearly as well.
The most important habit is to ask which question a number answers. Mean error addresses net direction. MAD addresses average distance. SD and limits of agreement address spread. Coverage percentages address how often errors fall within chosen bands. Correlation addresses co-movement. None alone answers whether a device will remain reliable after a person moves, after calibration drifts, or in a population absent from the study.
Once the metric panel is understood, the next problem is methodological rather than arithmetic. A result may still look impressive because the training and test data share subjects, because calibration and testing reproduce the same quiet condition, or because the BP range is too narrow to challenge the model. Interpreting the number therefore requires examining how the test was designed to produce it.