Who Is Being Tested—and In What Reality?
This chapter examines whether a medical-device validation study represents the people and conditions that matter. The practical judgment is not simply whether the study has enough participants. It is whether the participants’ capabilities and tasks match the intended users, whether the final interface and support materials are present, and whether the environment allows people to behave as they would in actual use. A participant count can be reported precisely and still support a weak safety conclusion if the test represents the wrong interaction.
The FDA’s human-factors guidance treats validation as evidence about intended users performing critical tasks with the final device under expected conditions. That standard makes representation and realism inseparable. A study with many participants in an artificial setting can miss a risk that a smaller, well-targeted group would encounter. A realistic room is not enough if the people are coached through the task or use an unfinished interface. The result must resemble the situation to which the conclusion will be applied.
A headcount is not representation
The most important participant criterion is whether the group represents the intended users. That sounds obvious until a device has more than one user population or when differences within a role can affect interaction. A professional caregiver, a patient, a family member, an installer, and a maintenance worker may all meet the broad description “user,” but they may perform different tasks and depend on different knowledge. Pediatric and adult users, or professional and lay users, may also need separate consideration when their characteristics change how they interact with the interface.
The reason to distinguish populations is not to create demographic categories for their own sake. It is to identify differences that can alter safe performance. A user’s age, education, literacy, sensory or physical limitations, occupational specialty, prior experience, and knowledge of the particular device may affect whether they notice an alarm, interpret a display, connect a component, or understand an instruction. If two groups perform different tasks or face different pathways to error, combining them into one average can hide the very variation that matters.
A sample can therefore be numerically large but evidentially narrow. Suppose a device is intended for both trained professionals and lay caregivers, but a study includes only professionals. The result may describe how those professionals performed under the tested conditions. It cannot establish how the caregiver population will interpret the interface, especially if the caregiver works at home, has less access to immediate support, or receives different training. The absence of errors among the studied professionals is not evidence that the unstudied population would have no errors.
The same issue arises within a single occupational group. If a task requires fine motor control, hearing an alarm, reading small information, or understanding a unit of measure, the relevant variation may not be captured by job title. A device intended for patients with a condition that can cause functional limitations should be considered in light of those limitations. If the design does not accommodate users who would otherwise be likely to use the device, the labeling should clearly explain what capabilities are necessary for safe and effective use. That statement is a boundary on the claim, not a substitute for examining whether a safer design is possible.
The FDA gives a general minimum of 15 participants for human-factors validation and says that distinct user populations should generally have at least 15 participants each.