How to Test a System That Changes by the Hour

A JITAI is not one scientific question, so it cannot have one universal trial design. This chapter connects the intervention’s conceptual model to the experiment that can test it: a fast repeated decision, a whole-program comparison, a combination of components, a slower adaptive pathway, or a coordinated human and social system. The practical result is a way to choose evidence that matches the JITAI’s timescale instead of using a prestigious trial name as a substitute for a clear question.

Match the experiment to the decision

Before choosing a design, specify what is supposed to change. The conceptual model begins with a meaningful distal outcome, identifies a proximal outcome that can change sooner, describes the mechanisms connecting them, selects intervention options, and states the conditions under which each option should help or fail. That sequence matters because an experiment can only answer a question that has been made explicit. “Does the app work?” leaves open whether the concern is a prompt, a threshold, a whole program, or an added coaching component.

Imagine a JITAI intended to improve oral self-care. Its distal aim might be sustained adherence to brushing recommendations over several weeks. Its proximal outcome might be whether brushing occurs in the next 12 hours. A motivational prompt is one possible option; a prompt to review feedback is another. A trial should not ask one design to answer every question about both options, their combination with coaching, and the long-term program. The timescale and the unit of the question determine the experiment.

When the question is about an option in a moment

A micro-randomized trial, or MRT, fits the fast timescale of a JITAI. It sequentially randomizes a person to different intervention options at many decision points, with relatively short intervals between randomizations. The same person can therefore contribute repeated contrasts: prompt versus no prompt, one type of prompt versus another, or delivery under one state versus delivery under another.

The MRT is useful when the question is close to the rule. Is a motivational prompt delivered an hour before a brushing window better than no prompt for brushing in the next 12 hours? Does a message focused on the benefits of brushing work better than one focused on the consequences of not brushing? Is the prompt more effective when the sensor indicates no brushing in the previous 12 hours than when some brushing has occurred? These are option-level and condition-level questions. Their outcomes should also be close to the decision, because a prompt delivered now cannot be cleanly evaluated only through a distant outcome measured weeks later.

The repeated randomization is not a technical flourish. It creates the variation needed to estimate whether an option changes a proximal response under a particular state. Without variation, a rule may always deliver the same option whenever the signal appears, leaving no comparable experience under the same condition. With variation, the study can examine whether the option is beneficial, for whom, and under what circumstances. The result can inform the rule, but it does not automatically establish that the complete JITAI improves a long-term clinical or behavioral outcome.

An MRT can also expose an interaction that an average program effect would hide. A prompt may help when the person is receptive and have little effect when they are overloaded. It may improve one proximal behavior but not another. It may work early in an intervention and lose its effect after repeated exposure. These questions are precisely about the changing relationship among state, option, and response. The MRT is suited to them because its randomizations happen on the same fast rhythm as the JITAI decisions.

When the question is about the whole program

A two-arm or multi-arm randomized trial answers a different question.