What the Experiments Actually Show: From First Ideas to Collective Diversity
To judge whether AI-assisted brainstorming converges too early, we need more than an impression that the outputs “sound alike.” The controlled human–LLM studies examined where diversity appears or disappears: across a pool of independent sessions, within one session as ideas accumulate, and after a different starting idea is supplied. The value of the evidence is not a universal score for humans or machines. It is a way to separate a collective distribution problem from an individual sequence problem.
A matched comparison, not a vibe
The first study placed human and LLM ideation under closely matched conditions. Ninety-nine human participants and ninety-nine simulated GPT-4o participants each generated ten fitness-product ideas, one at a time. The LLM condition used separate API calls for each idea while retaining the session’s previous responses in the conversation history, so the model could see what it had already proposed. That detail matters: the comparison was designed to reproduce serial ideation rather than give the model a single request for a list and compare the result with ten human acts of generation.
The researchers also needed a definition of “different.” Counting sentences or comparing their surface wording would be unreliable. A human may express two similar ideas in noticeably different language, making them appear far apart to a text-similarity measure. An LLM may use a uniform, polished template for ideas that differ in their underlying function, making them appear artificially close. The study therefore classified each product idea by meaning along three dimensions: the industry context it addressed, the psychological need it served, and the product form it took. It then used three measures: the number of unique categories covered, the number of unique combinations across those dimensions, and the average pairwise distance between ideas based on how many dimensions differed.
This design makes the measures answer different questions. Category coverage asks whether the pool reaches different broad areas. Category combinations ask whether those areas are being recombined into distinct concepts. Pairwise distance asks how far apart ideas are on the selected dimensions. None is a complete measure of creativity, but together they are more informative than a raw count of differently worded sentences. The category hierarchy was built through a structured abstraction and relabeling process, which was itself performed with GPT-4o; that makes the comparison more consistent while leaving a methodological dependency that must remain visible.
The main group-level result is striking. Across 990 ideas in each full condition, humans covered 28 categories and 206 unique category combinations. The default LLM covered 22 categories and 88 combinations. When the researchers repeatedly bootstrapped groups of ten participants, each contributing ten ideas, the human groups again spread farther: 26.29 versus 18.45 unique categories, 59.66 versus 39.15 unique combinations, and an average pairwise distance of 2.41 versus 2.04. The reported differences were highly significant.
At first glance, this seems to show that LLMs simply fixate more than humans. The within-participant results complicate that reading. For one participant’s ten ideas, humans and default LLM sessions were similar in accumulated categories: 12.68 versus 12.46. They were also similar in unique combinations, 8.37 versus 8.68, and average pairwise distance, 2.10 versus 2.11. None of those differences was statistically significant in the reported comparisons. The slopes of category accumulation were nearly identical as well: 1.035 for humans and 1.013 for LLMs.
These numbers do not mean that either condition explored maximally. Both were below the possible diversity ceiling, so both showed meaningful within-session constraint. They mean that, under this task and procedure, the LLM was not dramatically more repetitive than a human inside a single ten-idea session. The large difference emerged when the sessions were pooled.
That pattern is more informative than a single overall score because it identifies the level at which the distributions differ. If LLM sessions had accumulated fewer categories than human sessions and also produced less pairwise distance within each session, fixation would have been a plausible primary explanation. Instead, the reported within-session measures were close, while the pooled and first-idea measures separated sharply. The study therefore points toward a population-level concentration of starting points rather than simply a stronger individual tendency to repeat.
The bootstrapping design adds another useful check. Rather than relying only on one aggregate pool, the researchers repeatedly sampled smaller groups of participants and compared their distributions. The human advantage remained in categories, combinations, and pairwise distance. That makes the result less dependent on one unusual collection of sessions, although it still remains a result from one controlled task and one model condition. Statistical significance also does not say that every human idea was far from every LLM idea. It says that the groups’ distributions differed reliably under the study’s operational definitions.
The first-idea analysis isolates that difference. Before participants see their own previous responses, their first ideas reveal what each independent contributor reaches for initially. Human groups produced more diverse first ideas across all three measures: 14.51 versus 8.37 unique categories, 9.17 versus 5.88 unique combinations, and 2.41 versus 1.51 average pairwise distance. Those results support a difference in starting-point variation. Human participants were more distributed across the idea space from the first retrieval, while the independent LLM sessions drew more tightly around shared accessible regions.
First-idea diversity is not a judgment that the human starts were better. It is evidence that the contributors were less alike at the moment the search began. A narrow first-idea distribution can be appropriate if the problem has a sharply constrained answer, but it is a warning sign in exploratory work. It should prompt a second question: do the sessions remain narrow, or do they develop genuinely different consequences from similar beginnings? That is why the experiment does not stop at the first comparison.
The logic of the comparison is summarized below.
This is why the result should not be paraphrased as “humans are creative and AI is not.” The study finds a narrower claim. Individual LLM sessions can be about as diverse as individual human sessions under the tested sequential setup, yet a collection of human sessions covers more of the space because the people differ from one another. The collective gap is therefore evidence about the distribution of sessions, not a verdict on every single model response.
The seed test and the limits of the conclusion
A natural response to the first-idea result is to supply the model with more varied starting points. If people begin in different regions because their first associations differ, perhaps an LLM can be moved into those regions by giving it a human-generated idea. The second study tested exactly that possibility while keeping the rest of the procedure largely unchanged.
Each of ninety-nine simulated LLM participants received a different human participant’s first idea as a seed and then generated nine additional ideas. For a fair downstream comparison, the analysis excluded the shared first idea and compared ideas two through ten from seeded sessions with the corresponding ideas from default LLM sessions. If the missing ingredient were only a starting point, the seeded sessions should have remained more varied after the first response.
They did not. Default and seeded LLMs were nearly identical in total unique categories: 18.34 versus 18.35, with a reported p-value of .95. Their unique combinations were 38.22 versus 38.16, p = .89. Their average pairwise distances were both 2.07, p = .49. In this procedure, a human seed did not improve downstream diversity.
The result identifies an important difference between an entry point and a way of thinking. A seed changes the visible content of the prompt. It does not necessarily provide the persistent experiences, preferences, associations, or search policy that make one human’s subsequent ideas differ from another’s. If a human begins with an accessibility-oriented product idea, another person may develop it toward social support, low-cost distribution, or a different form of use because their background makes different implications salient. An LLM can receive the same first idea and still draw later continuations from essentially the same aggregate pattern as another session.
The seed result also helps separate the two bottlenecks. In the first study, clustered first ideas pointed to insufficient variation at the start. In the second, holding the start constant did not recover the variation that humans produced afterward. That suggests that human knowledge partitioning contributes not only different starting locations but also different trajectories through the space. A system that wants independent agents cannot reproduce that independence merely by copying one novel example into each context.
The comparison is useful precisely because it controls an appealing alternative explanation. One might say that LLMs looked homogeneous only because they were given homogeneous beginnings. The seeded condition gives each session a different human beginning and then asks whether the difference persists. It does not. The downstream results imply that the model’s subsequent generation process reabsorbed those varied starts into similar patterns. In practical terms, a prompt can import an example without importing the network of associations that made the example meaningful to a particular person.
This also clarifies what a successful partitioning intervention would have to do. It would need to alter more than the first sentence. It would need to provide distinct cues, information, or constraints that continue to shape what each session considers relevant as ideas develop. A persistent persona, a different retrieval context, a genuinely different model, or a separated interaction history might do that, but those possibilities were not tested by this seed comparison. The study establishes the limitation of a simple seed, not a complete theory of how to create independent AI ideators.
There are several reasons to resist making the conclusion larger than the experiment. First, the tested intervention was one seed followed by sequential generation. A seed combined with a different model, a distinct retrieval corpus, persistent memory, a richer persona, or a different interaction protocol could have another effect. The result says that this simple seed-and-continue procedure did not recreate downstream diversity; it does not prove that all forms of priming are ineffective.
Second, the participants and models were not interchangeable populations. The human data came from a controlled fitness-product task, while the LLM participants were simulated sessions using GPT-4o. The comparison improved control over instructions and sequence, but it was not a study of a real organization in which people and AI collaborate over time. Nor does it establish how other models, languages, domains, or prompt designs would behave.
Third, the measurements are proxies. Meaning-based categories are an improvement over naïve lexical similarity for this task, but they depend on a constructed taxonomy. Pairwise distance records differences along the selected dimensions, not the total value of an idea. A semantically distant product can be infeasible; a nearby idea can be the best response to the user’s actual constraint. Likewise, the later embedding analyses used to examine persona effects address structure in semantic space, not the hidden contents of a model’s mind.
Finally, the study emphasizes diversity because missing alternatives cannot be recovered after selection, but it does not settle the quality trade-off for every intervention. More categories do not automatically mean more novelty, usefulness, or feasibility. A task with a strong correct answer should not be forced into artificial variety. The evidence is strongest when the task is genuinely open-ended and when the reader asks a process question: did the system explore enough independent directions before narrowing?
The controlled evidence therefore gives us two signatures to carry forward. If ideas accumulate slowly within a session, look for fixation. If first ideas cluster across independent sessions, look for missing partitioning. If a new starting example fails to change what follows, the problem is not simply where the session began. These distinctions turn a general complaint about repetitive AI into a more precise diagnosis: identify the point at which independent exploration was lost, then judge an intervention against that point rather than against the final wording alone.