The Screening Failure Hidden Inside a Polished Pitch
This chapter diagnoses what the ifo study's 4–9% increase in screening error means inside the receiver's judgment. The focus is the failure pathway: average pitch quality rises, the spread of signals narrows, receivers become less accurate about expertise, and they express greater demand for additional evidence. Reading those outcomes together prevents the headline from becoming either a universal failure rate or an empty warning about “AI bias.”
What the 4–9% estimate actually means
The ifo study did not ask whether every AI-assisted pitch was worse. It compared the distribution of receiver judgments when senders could use ChatGPT with the distribution when they could not, using measures of error tied to the senders' observed expertise. The reported increase of roughly 4–9% describes the change in screening error in the pooled experimental settings and outcome definitions.
That qualification matters. The number is not a prediction that an employer will make 4–9% more bad hires, nor that an investor will lose 4–9% more deals. It is an estimate of how much less accurately receivers in this experiment identified expertise when the pitch condition included ChatGPT assistance. The magnitude varies with the error measure and the regression specification, so the direction and interpretation are more stable than any one presentation of the percentage.
The study uses two related ways to capture inaccuracy. One looks at the variance of receivers' probability beliefs. If a receiver is uncertain about whether a sender is an expert, their probability tends toward the middle. The other calculates the squared difference between the receiver's predicted expertise and the sender's actual expertise. The first measure describes uncertainty in belief; the second describes distance from the observed classification. The measures are conceptually linked but not identical, which is why the authors report both.
The results show increased error across the main outcomes, with the log specifications giving the clearest percentage interpretation. In levels, the squared-error estimate is less precisely detected in some specifications, but the broader contingency-table and variance evidence points in the same direction. That combination is important for an evidence-based reading: the conclusion does not rest on a single convenient transformation.
The experiment also rules out a simpler explanation. If ChatGPT merely made receivers overestimate the treated group, the average belief that a sender was an expert should have risen. It did not. Receivers assigned senders about a 45% chance of being experts in both the ChatGPT and non-ChatGPT conditions, while the task contained an actual expert share of 50%.
The stable average hides a change in the allocation of confidence. Receivers did not become broadly convinced that AI users were experts. They became less able to tell which individual senders were experts. In statistical language, the problem is not only a shift in the mean belief; it is a deterioration in the variation that helps classify cases. In ordinary language, the receiver's overall guess remained similar while the match between the guess and the person became less reliable.