Who Pays for Popularity Bias? Three User Trajectories Through the Loop
This chapter examines how the recommender feedback loop is distributed across users rather than reported as one system average. The dynamic study separates people by their historical relationship to popular and less-popular items, then tracks calibration, ranking accuracy, long-tail exposure, and recommendation diversity over repeated cycles. The goal is not to turn those groups into permanent identities, but to see how the same popularity structure can produce different forms of future-choice loss.
Three profiles inside the same catalog
To study popularity bias, the researchers first divide items into a head and a tail. The head contains the most popular items, identified from rating frequency; the tail contains the remaining, less frequently rated items. In the reported framework, the boundary follows a Pareto-style rule in which the items accounting for the top 20 percent of rating frequency are treated as head items and the rest as tail items. This is an analytical boundary, not a claim that the twentieth percentile marks a natural social divide. Change the catalog or the threshold and the measured groups can change with it.
Users are then grouped by the proportion of their historical interactions with head items. A user with a high proportion is Popular-focused. A user with a low proportion is Niche-focused. Users between one standard deviation above and below the observed mean are Diverse-focused. The use of the observed mean and standard deviation makes the cutoffs data-adaptive, rather than imposing the same numerical threshold on every dataset. It also means that the groups need not be equally large. The labels describe a profile of recorded interaction with popularity; they do not describe a person's age, gender, personality, political identity, or stable essence.
The reported profiles make the asymmetry visible. In the MovieLens experiment, Popular-focused users had 93 percent of their interactions with head items, Diverse-focused users 84 percent, and Niche-focused users 70 percent. In the Personality 2018 dataset, the corresponding proportions were 98, 86, and 68 percent. The niche-oriented groups also had much larger profiles: in the MovieLens data, their average profile size was reported as 380 items, compared with 49 for Popular-focused users and 148 for Diverse-focused users. In the Personality data, the contrast was 1,554 items for Niche-focused users, 71 for Popular-focused users, and 444 for Diverse-focused users.
These numbers do not mean that a niche-oriented user is necessarily receiving a niche-oriented recommendation list. They describe the starting history against which the recommendation output is compared. That distinction is crucial. A system can appear to personalize well for a user who prefers popular items because its output is already close to the catalog's dominant region. A system that moves a niche-oriented user's output toward the head may still produce items that are individually plausible, yet it is less calibrated to the user's original distribution. The relevant question is not only whether the algorithm can rank something the user might like, but whether the ranking preserves the shape of what the user has shown they value.
The segmentation also makes fairness concrete without reducing fairness to protected attributes. The study is asking whether a recommendation process aligns differently with users who occupy different positions in the popularity distribution. That is a user-side question. There is also an item-side consequence: if less-popular items receive less exposure, creators and catalog regions outside the head have fewer opportunities to accumulate interaction evidence. User calibration and item exposure are linked by the same ranking decisions, but they are not identical outcomes. A list can be poorly calibrated for a niche user while still giving some tail items exposure elsewhere, or it can concentrate exposure so strongly that both user breadth and catalog coverage decline.
Six windows onto drift
No single metric can show the entire change. The study therefore uses several measures that operate at different levels. The first pair concerns popularity alignment and calibration.
Group Average Popularity difference, or ΔGAP, compares the average popularity of items in a group's historical profile with the average popularity of the group's recommendation lists. It is expressed as a relative difference between recommended and historical popularity. A value near zero indicates closer alignment. A positive value indicates that recommendations have shifted toward more popular items than the user's history. In this context, a rising ΔGAP is evidence of calibration drift toward the head, not simply evidence that the list contains popular items.
Average Popularity of Recommended Items, or APRI, looks more directly at the popularity of items in the top-N recommendations. Lower values generally indicate less reliance on highly popular items and therefore more opportunity for less frequent content. APRI is useful as an exposure measure, but it should not be read as a pure quality score. A lower average popularity could reflect valuable discovery, irrelevant items, or a catalog with a different popularity distribution. Its meaning depends on the user's preferences and the other metrics around it.
The next window is conventional ranking quality.
Normalized Discounted Cumulative Gain, or nDCG, evaluates whether more relevant items appear near the top of the ranked list. It answers a legitimate question: how consistently does the ranking place items that match the user's measured relevance near the top? But nDCG does not reveal whether those items are concentrated in the head, whether a user's exposure becomes less varied, or whether a different group is losing more of its initial breadth. A stable or improving nDCG can therefore coexist with a shrinking choice environment.
The remaining measures are beyond-accuracy indicators.
Average Percentage of Long-Tail items, or APLT, measures the share of a user's recommendation list that belongs to the tail. It is a direct window onto individual exposure to less-popular content. Entropy measures how evenly items are distributed across aggregated recommendation lists. It asks whether exposure is spread across many items or concentrated on a few. Long-Tail Coverage, or LTC, measures the share of the available tail catalog represented across the recommendations. APLT is closer to what one user sees; entropy and LTC describe broader exposure patterns. The distinction matters: an item can appear in the system somewhere without appearing often enough for a particular user, and a user can receive varied items while the system as a whole still neglects most of the tail.
The measures also differ in what they cannot establish. A higher APLT does not prove that a user experienced meaningful choice. Higher entropy does not prove that exposure was useful or comprehensible. More coverage does not prove that the catalog's political, emotional, or cultural range was represented fairly. These metrics make different kinds of narrowing visible; they do not translate automatically into autonomy, welfare, or social tolerance.
The reported trajectories show why the combination matters. Across the two datasets and three collaborative-filtering models, Niche-focused users generally began with the strongest long-tail exposure, highest entropy, and greatest long-tail coverage. That starting advantage is intuitive: their histories already contain more less-popular items. Over ten simulated iterations, however, their advantage usually contracted. Their calibration drifted most rapidly toward popular items, and their ranking accuracy often declined proportionally more than that of other groups. Their absolute nDCG could remain high in some configurations, but their trajectory was worse relative to where they began.
The pattern is visible in a concrete MovieLens example. Under VAECF, the Niche-focused group's ΔGAP rose from 34.537 at baseline to 44.081 at the tenth iteration, while APRI rose from 0.139 to 0.162. Its nDCG changed only slightly, from 0.624 to 0.622, but APLT fell from 0.814 to 0.761 and entropy from 0.812 to 0.778. Long-tail coverage increased slightly in that particular configuration, from 0.3282 to 0.3357. The example resists a simple story: popularity alignment worsened and individual exposure breadth declined, while one system-level coverage measure moved in the opposite direction. That is exactly why the metrics should be read together rather than collapsed into “the algorithm got worse.”
In the Personality dataset, the same broad tension appears more sharply for several models. For Niche-focused users under VAECF, ΔGAP rose from 296.065 to 339.558 and APRI from 0.644 to 0.705. APLT fell from 0.526 to 0.437 and entropy from 0.411 to 0.378, while nDCG rose from 0.640 to 0.657. Long-tail coverage increased from a very small base, which makes the large percentage change difficult to interpret as a broad recovery. Here again, ranking accuracy improved slightly even as popularity alignment and individual breadth moved in an unfavorable direction. A system that reported only nDCG could describe this as progress; a user-centered exposure view would describe a more complicated transition.
Popular-focused users occupy the opposite position. Their historical behavior is already concentrated on head items, so the recommender's mainstream tendency often looks well calibrated to them. In the reported results they frequently gain or preserve ranking accuracy. Yet their APLT is low, and in the Personality dataset it is zero for all models at baseline and after ten iterations. Their experience can therefore be accurate in the narrow ranking sense while remaining confined to a small and popular part of the catalog. The system is not necessarily failing to satisfy their immediate preference; it is failing to create much evidence that they should encounter anything else.
Diverse-focused users begin between those extremes. Their importance is not that they represent an ideal group, but that they show how a balanced profile can drift. The results describe them gradually losing calibration and diversity advantages and moving toward patterns that resemble Popular-focused recommendations. Their middle position does not protect them from the loop. It can instead make the change easy to miss: they may retain acceptable accuracy while their exposure becomes less varied over time.
The study reports statistically significant differences between the most distinct groups in several comparisons, but significance labels do not replace magnitude or mechanism. The meaningful pattern is the combination of group order, direction of change, and metric family: niche users lose much of their initial exposure breadth; popular users remain close to the head; diverse users tend toward the same concentration. The results also contain exceptions. Under VAECF on MovieLens, long-tail coverage for each group increased slightly, while other models showed declines; some absolute accuracy values for niche users remained strong. These exceptions do not erase the overall trend, but they prevent it from becoming a deterministic law of every recommender.
The broader lesson is about who pays for a system's apparent stability. Popular-focused users may receive the accuracy gains that make the recommender look successful, but at the cost of remaining in a narrow mainstream region. Niche-focused users may begin with the richest and most informative profiles, yet see their calibration, novelty, and long-tail exposure erode. Diverse-focused users may lose the very breadth that initially distinguished their histories. The loop does not impose one identical harm; it reallocates the ability to encounter alternatives.
That is why future choice must be evaluated as a trajectory. A one-shot list can look reasonable for every group. Repeated lists can make the groups diverge. The next task is to combine this evidence into one model of what is changing: not merely the accuracy of recommendations, but the state of the exposure environment in which future preferences are formed and measured.