The Delegation Trap: Why AI Does Not Carry Responsibility Away
This series examines responsibility in AI-mediated work: what happens when an AI recommends, constrains, or helps make a decision that affects people. The immediate question is not whether a system produced an output, nor whether a human was somewhere in the workflow. It is whether anyone retained meaningful discretion, understanding, and causal influence after the AI entered the process. That distinction helps explain a result that initially seems contradictory: an AI can make a human look more responsible in one arrangement and less responsible in another.
The difference matters wherever AI is used in lending, hiring, insurance, compliance, safety, or other decisions with consequences beyond the screen. Organizations often describe these arrangements as delegation. A person delegates part of a decision to a model, or an automated system hands a problem back to a human overseer. The language suggests that responsibility travels with the task. The evidence discussed here suggests something more complicated. Delegation changes the whole action system: the roles, the available choices, the information people can access, and the points at which an outcome can still be changed.
A decision can be delegated without responsibility being transferred
Consider two superficially similar situations. In the first, a bank employee works with an AI system to assess loan applications. The team is later found responsible for discriminatory lending. The employee did not build the model, and the scenario does not say that the employee made a particular mistake. Still, observers may regard the human as the more responsible teammate because the AI appears to be operating within rules, objectives, and technical constraints. The human seems to be the party who could have questioned an output, interpreted the context, or taken ownership of the final decision.
In the second situation, a person oversees an autonomous delivery drone. An intelligent support system decides which interventions are safe and grays out most of the available actions. The drone may become safer and the person may make decisions faster. But if only one action remains, the overseer may no longer feel that they caused the crash or could have prevented it. The person is still present, yet the connection between their choice and the outcome has weakened.
These are not opposite findings about the same treatment. They are different changes to the action system. The first adds an AI teammate while leaving the human as the apparently discretionary member of the pair. The second narrows the human’s action space until the remaining role may be largely ceremonial. In both cases, the important question is not “Was AI involved?” It is “What kind of control did each participant retain?”
That question also separates three ideas that are often collapsed into the word responsibility.
- Output responsibility concerns which system produced a recommendation or action.
- Role responsibility concerns what a person or organization was formally assigned to do.
- Moral responsibility concerns whether an actor understood the relevant situation and had enough control or opportunity to affect the outcome to be fairly held accountable.
An AI can produce an output without being the only meaningful cause of what followed. A human can occupy an oversight role without having a genuine opportunity to prevent the harm. And an organization can assign a duty to a frontline employee without giving that employee the authority or information needed to discharge it. The visible location of the final click is therefore an unreliable guide to the full distribution of responsibility.
The two findings that create the paradox
A study of AI–human teams gives the first half of the picture. Across four experiments, researchers examined how people assign responsibility after a morally consequential mistake when a human worked either with an AI or with another human. In the first study, participants imagined themselves processing loan applications with one of those two teammates. After learning that the team had been responsible for discriminatory lending, they rated how much responsibility they themselves carried.
Participants paired with AI attributed more responsibility to themselves than participants paired with a human colleague: 47.81 versus 40.81 on a 0–100 scale. The difference was modest but statistically significant. The result is notable because several ordinary predictions point the other way. People often distrust algorithms after failures, and people commonly protect their own self-image by shifting blame elsewhere. An AI might appear to be an especially convenient target. Yet in this team setting, the human became the more plausible bearer of responsibility.
The researchers call this pattern AI-Induced Human Responsibility, or AIHR. The name describes an attributional effect, not a rule that humans are always normatively responsible for AI-related harm. The study’s scenarios deliberately left the causal contribution of the two teammates ambiguous. The result shows how people resolve that ambiguity psychologically: when the AI is framed as a teammate, responsibility can be pulled toward the human rather than pushed toward the machine.
The drone-oversight study gives the second half. Participants monitored an autonomous drone through a simulated interface. In different conditions, they could select six, four, two, or one action when a critical situation occurred. When only one action was available, participants felt less morally responsible for a crash than those who had more options. They also reported a weaker causal connection to the crash and less knowledge of how it could have been prevented. The restriction improved decision accuracy and shortened decision time, but it also weakened the perceived basis for holding the overseer responsible.
The lesson is not that more options are always better. Some options may be dangerous or irrelevant, and removing ineffective choices can improve safety. The lesson is that a remaining option must be meaningful: it must allow the person to alter or prevent the outcome, and the person must understand enough of the situation to know why that choice matters. Otherwise, “human oversight” can leave a person formally in charge while depriving them of the conditions that make responsibility intelligible.
Together, the studies expose the trap in a simple account of delegation. If the AI is treated as a constrained teammate, the human may inherit perceived responsibility because they appear to be the only actor with legitimate discretion. If the human is constrained so severely that their choice cannot change the result, perceived responsibility may recede. Responsibility does not move in a single direction when AI enters a workflow.
A working model: autonomy, choice, knowledge, and causality
The most useful way to organize the issue is to treat responsibility as a relationship among several conditions rather than as a property of the AI or the human alone.
First is perceived autonomy. In the AIHR research, autonomy means more than the ability to produce an independent-looking output. It concerns whether an actor is seen as free and authorized to choose within the situation. An AI may generate sophisticated recommendations, but people may still regard it as constrained by human-written objectives, training data, guardrails, and organizational instructions. The human teammate, by contrast, is expected to interpret the recommendation, challenge it, override it, or accept ownership of the decision. That structural expectation makes the human look more accountable even when the AI materially shaped the outcome.
Second is meaningful choice. An overseer needs more than a button labeled “approve.” The available action must matter to the outcome. If every remaining action leads to the same harm, or if the system has already removed the only interventions that could have prevented it, the person’s apparent choice is a pseudo-choice. The person may be present at the decisive moment but not genuinely able to decide.
Third is epistemic access: understanding what is happening, why the system is recommending an action, what its limits are, and how the harm might be prevented. A person cannot fairly be held responsible for failing to correct a problem they had no reasonable way to recognize. More information is not automatically enough; the information must help the person understand the decision situation and the system’s failure modes.
Fourth is causal contribution. The person’s action must make a difference to the outcome, or at least they must have had a real opportunity to prevent or alter it. Causal contribution is not identical to physical proximity. A human who clicks after an AI has fixed the available choice set may be closer to the outcome in time but less causally influential than a developer or organization that designed the system and its constraints.
These conditions can diverge. A human may have formal authority but poor information. They may understand the situation but lack a timely override. They may have several options, but none that can change the result. Conversely, an AI may have substantial operational influence while being perceived as low in autonomy because its role is tightly constrained. The result is a gap between perceived responsibility and actual control.
This model does not solve every accountability question. It does, however, prevent two common errors. The first is to treat an AI’s involvement as an accountability sink: once a model is named, the human and organization are assumed to have delegated the duty away. The second is to treat a human’s presence as sufficient proof of responsibility: if a person was in the loop, they are assumed to have been able to control the result.
The sources instead support a more careful judgment. Ask who could understand the failure, who could choose among outcome-relevant alternatives, who could override or escalate, and who shaped the conditions under which the decision was made. Those questions keep the human–AI arrangement at the center while recognizing that responsibility can extend beyond both the immediate user and the model.
The next step is to examine the first mechanism in detail: why an AI–human team frame can make the human appear more responsible precisely because the AI is seen as less autonomous. That shift is not a contradiction of the oversight result. It is the consequence of asking a different question about control.