When the Chat Starts Looking Back at You
The stronger claim in human–AI personality research is not that users prefer a chatbot. It is that a conversation may alter how users describe themselves. This chapter examines the pre- and post-conversation test designed to detect that movement: participants rated their own personality traits, talked with a default GPT-4o chatbot, and rated themselves again. The result is useful only if we keep its target precise—a short-term change in measured self-concept, under particular topic and model conditions, rather than a diagnosis or a permanent personality rewrite.
Measuring a moving self-concept against a chatbot
Self-concept is the way a person understands and describes themselves. It includes beliefs about abilities, appearance, values, and personality. The study focused on the last of these: how strongly participants said that a selected set of positive and negative traits applied to them. That is a narrower object than “personality” in everyday speech. A personality trait usually implies a pattern that can be relatively stable across situations; a self-concept rating is a person’s current interpretation of that pattern. The two can correspond, but a change in the rating does not automatically mean the underlying disposition has changed.
The design was built to capture a within-person movement. Researchers recruited 110 adults in the United States through the Connect crowdsourcing platform. After excluding 18 people who did not complete the study, 92 remained, with 46 assigned to personal topics and 46 to non-personal topics. Participants rated twenty traits on sliders from 0, meaning a trait did not apply at all, to 100, meaning it applied extremely. They completed that scale before the conversation and again afterward.
The conversation used the same GPT-4o model for everyone, with the API’s default parameters and identical system prompts. Participants were randomly assigned to one of two topic environments. Personal topics concerned their lives, emotions, identity, or experiences—for example, what they would choose if time and money were no object. Non-personal topics concerned public, objective, or general subjects, such as why cats are popular on the internet or how historical events shaped modern technology. The chat was text-only, had a five-minute minimum and a fifteen-minute maximum, and required at least ten turns. Across the retained sample, the average conversation lasted nine minutes and included about 20.9 turns.
The researchers needed a reference point for the AI as well as for the humans. They prompted GPT-4o through the API to complete the same twenty-item trait scale one hundred times, using the model settings and system prompt associated with the participant-facing interaction. They averaged the repeated responses for each item to create the chatbot’s measured profile. This does not mean GPT-4o possessed the resulting traits. It means that, under the study’s fixed configuration, the model produced a repeatable-enough profile that could be used as a comparison target.
The comparison itself is easier to understand as a distance question. Imagine each person’s twenty ratings as a point in a twenty-dimensional space: one coordinate for generosity, another for cheerfulness, another for confidence, and so on. The researchers used Manhattan distance, which adds up the absolute difference on each trait between two profiles. A large distance means that the two patterns of ratings differ across many dimensions or differ substantially on a few of them. A smaller distance means the patterns are more alike.