Overview

In a fictional Harborview flood-budget task, 380 UK participants saw an expert recommendation and then read fixed AI counterarguments without factual claims or an action recommendation. Reliance was measured as moving the approval decision toward the AI after accounting for the initial choice; 56% of initial approvers moved toward the AI's anti-expert position, versus 10% of initial rejectors moving toward approval. Warmth and contextualization did not significantly change reliance. Because the scenario had no objective answer or real stakes, deference was not proof of harmful overreliance.

Learning outcomes

The initial decision strongly predicted the final position (β = .498), more than the tested presentation treatments.
Among initial approvers, 56% moved toward the AI's anti-expert position, versus 10% of initial rejectors.
The fictional, consequence-free task showed deference to AI, not proof that the resulting decision was wrong or harmful.

Keywords

Conversational AIRelianceDeferenceExpertsUncertaintyOverreliance

When AI Advice Beats an Expert

Political petition signing is one form of action: a person chooses to support a cause. Everyday decision support raises a different question. What happens when a person has little prior knowledge, sees an expert recommendation, and then encounters an AI assistant arguing against it? This chapter examines conversational AI reliance in that situation. The key judgment is not whether the user changes their mind in the abstract, but whether they defer to the AI when another source of judgment is present—and whether that deference should be called harmful at all.

A Low-Knowledge Decision Without a Known Right Answer

The experiment placed 380 UK participants in a fictional scenario about Harborview, a city facing worsening floods. The city was considering StormShield, a flood-barrier system that used sensors to anticipate sudden changes in water levels. Experts had assessed the project and recommended a risk budget, but allocating those resources would leave fewer funds for other public services. Participants first decided whether to approve the expert-recommended budget and reported how confident they were in the expert evaluation.

The fictional setting was deliberate. In political persuasion research, participants often arrive with existing views that can be measured and shifted. That makes it difficult to separate the effect of a conversational design from the strength of someone’s prior commitment. A made-up city and infrastructure project gave participants little reason to bring a settled political identity or a detailed factual position into the interaction. It also made the task resemble a common use of conversational AI: asking an assistant to explain an unfamiliar problem when the user is not sure how to evaluate the answer.

The design removed another source of confusion. The expert assessment and the AI assistant’s counterarguments contained opinions rather than factual claims. The assistant was given the same arguments for every participant and was instructed not to recommend a course of action or openly tell the user to reject the experts. Its role was to present counterarguments. This meant the experiment was not primarily testing whether the AI knew more facts than the experts. It was testing how people responded when an AI presented an opposing judgment in different conversational forms.

The researchers obtained each participant’s work domain, field of study, or hobbies before the scenario. Those details were used for one of the presentation manipulations: a contextualized assistant could explain the counterarguments through a background the participant recognized. A separate manipulation changed warmth. Warm responses used a friendly greeting, praise for the question, and selected emojis; neutral responses kept the same basic answer structure without those additions. The argument itself was held as constant as the design allowed. This separation is important because it gives contextualization and warmth a specific meaning in this study: they are features of presentation, not synonyms for all possible forms of personalization.

Published by DeepMox

FAQ

What did reliance mean in the Harborview experiment?

It meant moving the post-interaction budget decision toward the AI's position after an expert recommendation was available, while accounting for the participant's initial decision.

What does the 56% switching figure in Harborview mean?

It means 56% of participants who initially approved the expert recommendation moved toward the AI's anti-expert position; 10% of initial rejectors moved toward approval.

Why was the Harborview scenario designed without a known right answer?

It separated responses to AI presentation from factual superiority and prior political commitments. The fictional, opinion-based setting reduced those confounds but prevented researchers from judging whether a decision was correct.

Previous10 minConversational AI persuasion: Strategies for turning support into action

Next9 minConversational AI persuasion: Why personalization can backfire

Conversational AI persuasion: Following AI against an expert

When AI Advice Beats an Expert

A Low-Knowledge Decision Without a Known Right Answer

FAQ