The Core Test: Did Better-Looking Pitches Carry Less Information?

The model says that AI changes screening through the relationship between expertise and signal boost. The experiment makes that relationship visible through separate measures of pitch quality, variation, receiver belief, error, and demand for more evidence. This checkpoint connects those pieces into one judgment: a pitch can improve in apparent level while losing information if the difference between expert and non-expert outputs becomes harder to read.

From signal cost to an observable test

The theoretical mechanism begins with a simple idea. A pitch is informative when the people who possess the relevant expertise can produce a strong one more reliably or at lower cost than people who do not. AI changes that production condition. The tool may lift the visible quality of many messages, but the screening question is whether it changes the distance between the groups in a way the receiver can use.

The experimental design supplies the comparison the theory needs. Senders worked in domains where they did or did not have prior expertise. They produced pitches under conditions with and without ChatGPT assistance. Receivers judged the messages, estimated the likelihood that the sender was an expert, and rated the pitch. Because the study retained a measure of actual expertise, researchers could ask whether a receiver's confidence tracked the underlying classification.

This is a stronger test than asking whether readers preferred the AI-assisted version. Preference captures an impression. Screening accuracy captures whether the impression separates the cases that matter. A higher score can be useful evidence if it rises more for experts. The same higher score can be a weaker signal if it rises for experts and non-experts alike.

The model also predicts that the receiver's response will show up outside the main rating. If the pitch becomes less trustworthy as evidence, receivers may become more uncertain, make more classification errors, or seek an additional signal. Those outcomes are not decorative additions to the study. They are ways of checking whether apparent improvement in the message translated into better information for the decision.

The four-part pattern: level, dispersion, accuracy, validation

The pooled ifo results form a pattern rather than a single number. When senders could use ChatGPT, the evaluator-rated level of the pitch rose by about 4–6 percent in the reported specifications. The natural-language sophistication measure rose more strongly, around 32 percent. On the surface, the messages looked better.

At the same time, the study estimates that screening errors increased by roughly 4–9 percent. The exact value depends on the error measure and specification, but the direction is consistent across the main accuracy results. This is the central contrast: the receiver saw a higher-level signal but became less accurate at identifying the sender's expertise.

The average belief did not simply shift upward. Receivers assigned senders about a 45 percent chance of being experts in both the ChatGPT and non-ChatGPT conditions, while the true share of experts in the task was 50 percent. The loss was not mainly a story of receivers assuming that AI-assisted senders were all more capable. It was a story of separation. Receivers became more uncertain about which particular senders were experts.

The distribution of pitch quality moved in the same direction as the accuracy problem. ChatGPT compressed signal variation.