When AI Joins the Team, Why the Human Becomes the Accountable One
This chapter focuses on a specific feature of AI-mediated work: the AI is not described as a detached tool but as a teammate sharing a decision with a human. The question is why that framing can make the human appear more responsible after a failure, even when people are accustomed to blaming algorithms. The answer lies less in how intelligent the AI seems than in who appears to have discretion inside the team’s structure.
That distinction matters because “AI-assisted” can describe very different arrangements. An employee may use a model as one input among several, accept a recommendation automatically, or work with a system that is presented as a co-decider. These arrangements can produce similar outputs while inviting different judgments about who could have questioned the process. The AI–human team research isolates that social and structural judgment through a controlled comparison.
What the loan-processing comparison changed
The first experiment used a simple but consequential scenario. Participants imagined working in a bank’s credit and loan department. Their team assessed loan applications, compared requested amounts with documents such as identity records and payslips, and decided whether to accept or change a proposed loan amount. The team then learned that the bank had received discriminatory lending complaints and that an internal audit had found the team responsible for the mistake.
The crucial manipulation was the identity of the second team member. In one condition, the participant worked with an artificial intelligence system. In the other, the participant worked with a human colleague. The task, the later complaint, and the team’s stated scope of work were otherwise held constant. Participants then allocated responsibility between themselves and the other team member on a 0–100 scale.
The comparison was not a test of whether one particular algorithm was accurate. The researchers did not give the AI a special failure mode, nor did they tell participants that the human had ignored a warning or that the model had malfunctioned. The causal contribution of each teammate remained ambiguous. The experimental question was narrower: when the same joint task is framed as AI–human rather than human–human, does the identity of the teammate change where people place responsibility?
It did. Participants in the AI–human condition attributed more responsibility to themselves than those in the human–human condition: an average of 47.81 compared with 40.81 on the self-responsibility scale. The difference was statistically significant, with a reported effect size of d = .24. It is not a gigantic shift, and the study does not claim that every participant reached the same judgment. But it is enough to reverse the most immediate intuition that a morally harmful algorithmic outcome will simply produce blame directed at the machine.
The finding is called AI-Induced Human Responsibility, or AIHR. The name should be read carefully. It describes what people attributed in the experiment; it does not establish that the human was in fact the principal cause of discriminatory lending, or that assigning more responsibility to the human would be fair. The scenarios offered no evidence that one teammate had done more than the other. In that sense, the result reveals a default way of resolving ambiguity, not a final answer to an accountability investigation.
Why is the default surprising? One reason is algorithm aversion. People often react negatively when an algorithm makes an error, especially in a domain with moral stakes. Another is self-serving bias: people generally have incentives to protect their self-image and avoid blame for a bad outcome. An AI might seem to offer an especially convenient target because it is opaque, nonhuman, and already regarded with suspicion in many settings.
But those expectations come largely from situations in which a person receives or uses an algorithmic output as a separate object. The AIHR scenario does something different. It says that the human and AI are a team with a shared work responsibility. That wording turns the problem from “Who produced the bad recommendation?” into “How should responsibility be partitioned inside a joint action system?” Once the question changes, people look for the teammate who had legitimate discretion rather than simply the component that generated the output.
Why a team frame changes the attribution problem
A human–human team gives an observer no automatic reason to favor one member when the scenario supplies no further information. Both members are recognizable moral agents. Either could, in principle, have misunderstood the application, failed to challenge a decision, or acted within a role that shaped the result. A random or roughly balanced allocation is therefore plausible when the facts are silent.
An AI–human team introduces an asymmetry even when the task description says the two members are jointly responsible. The AI may be able to analyze documents, identify patterns, and produce a recommendation. Yet observers also know that its objectives, training, guardrails, and operating permissions were specified through human decisions. The system’s output can be complex without being seen as an act of independent moral judgment. The human teammate is consequently assigned a different kind of role: the person expected to contextualize the output, notice when a rule misfires, and decide whether to proceed.
This is a structural inference. It does not require the observer to believe that the human is technically more capable. It requires only the belief that the human is the party who can legitimately say no. In ordinary organizations, people are accustomed to treating humans as bearers of workplace obligations and machines as constrained instruments or implementers. A team frame makes that contrast salient rather than hiding it behind the broad label “AI-assisted.”
The process can be represented as a sequence:
The diagram is not a claim that every organization implements this sequence literally. It makes the psychological logic visible: the organization sets the AI’s boundaries, the AI contributes to the shared task, and the observer then searches for the person who had authority to interpret or override the system. The decisive link is not “AI produced output, therefore human is guilty.” It is “AI appears constrained, therefore the human appears to retain the relevant discretion.”
That distinction also explains why “the AI was only advisory” can have an unexpected effect. Calling a system advisory may be intended to preserve human accountability: the employee is told that the recommendation is not a decision and that they remain responsible for the result. Yet if the interface and workplace norms make the recommendation difficult to challenge, the employee may be practically constrained while still being perceived as the accountable party. Role language and actual authority can therefore point in different directions.
Perceived autonomy is not perceived intelligence
The proposed mechanism is perceived autonomy. In ordinary language, autonomy can sound like intelligence, initiative, or the ability to operate without a person typing each instruction. In the AIHR research, the relevant idea is narrower and more useful for responsibility: whether the actor is perceived as having meaningful freedom and authority to choose within the situation.
An AI can be highly capable and still be perceived as low in this form of autonomy. It may compare thousands of records, identify a statistical pattern that no employee could see, or generate a fluent explanation. None of those abilities alone shows that the system was free to define its own objectives, interpret the moral significance of a case, or depart from the constraints imposed by its deployment. A capable system can still look like a constrained implementer.
The human teammate occupies the opposite position. Even if the person depends heavily on the recommendation, observers may assume that the person could ask for more information, recognize a contextual exception, refuse the recommendation, or escalate the case. Those assumptions make the human the apparent bearer of discretionary judgment. Responsibility is pulled toward the human not because the AI has no effect, but because the AI’s effect is interpreted as operating inside a structure the human is expected to supervise.
The researchers’ second study supports this mechanism. Participants evaluated scenarios involving either high harm—falsified financial information and loans beyond a client’s ability to repay—or low harm, such as a filing error that forced another team to work overtime. They again compared AI–human with human–human teams and measured both responsibility and perceived autonomy. The human received more responsibility in the AI condition at both harm levels. Perceived autonomy also differed strongly by team type, and the statistical analysis found a significant indirect pathway from team type through perceived autonomy to responsibility.
The mediation was partial, not complete. The team-type effect remained after perceived autonomy was included, so autonomy is an important mechanism rather than a demonstrated exhaustive explanation. That qualification matters. Psychological judgments rarely have a single switch. Familiarity with workplace roles, beliefs about control, expectations about oversight, and the wording of a scenario may all contribute. The evidence supports autonomy as a central route through which the team frame changes responsibility, not as the only route in every context.
What the mechanism is not
The study also tests two explanations that might otherwise be mistaken for autonomy. The first is threat to self-worth. As AI becomes associated with job replacement, a human paired with it might feel that their own value is being challenged and compensate by taking ownership of mistakes. In the third study, however, the AI condition did not significantly increase perceived self-threat, and self-threat did not mediate the higher human responsibility attribution. In the analysis, greater self-threat was associated with lower responsibility attribution rather than explaining the AIHR pattern.
The second is mind perception: the idea that people assign responsibility according to whether an AI seems to understand, remember, plan, or distinguish right from wrong. In the fourth study, participants did perceive less mind in the AI teammate than in the human teammate. Yet mind perception did not mediate the responsibility effect. The AI could be regarded as less mentally capable while the human responsibility shift was explained more directly by the perceived structure of action.
This is a useful distinction for evaluating real systems. A fluent conversational agent may appear highly intelligent or even humanlike, but that appearance does not tell us whether people think the system had authority to choose. Conversely, an unremarkable scoring system may be treated as highly consequential if an organization gives it practical control and makes employees follow it. Intelligence and autonomy can travel together, but they are not the same variable.
The null results should not be overstated. They rule out the tested mediation pathways in the study designs; they do not prove that self-worth or mind perception never matters. Different settings could make identity threat more salient, or a visibly agentic system could change how people interpret autonomy. The safer conclusion is comparative: within these AI–human team scenarios, perceived freedom to act within the work structure explains more of the responsibility shift than perceived mental capacity or self-worth threat.
That conclusion changes how a manager should read an AIHR-like result. If employees appear to be taking disproportionate responsibility, the immediate question should not be whether they are unusually conscientious or whether the model is unusually intelligent. It should be whether the work design has made the employee the only actor who appears authorized to exercise judgment. If so, the organization may be creating a responsibility asymmetry by pairing a human with a system whose constraints are invisible in the final decision but obvious in the attribution of blame.
The next issue is the other side of autonomy. If people assign responsibility to the human because the human appears free to choose, what happens when an interface removes the choices that could make oversight meaningful? The controlled drone study provides that test. It shifts attention from who appears autonomous in a team to how much autonomy the human actually experiences at the moment of intervention.