Why Agents Still Miss the Best Fit

Agentic shopping systems can often find something that resembles a request, yet still fail to curate a product that fits the person. This chapter diagnoses that gap by separating product discovery, link attribution, evidence extraction, constraint handling, and recovery. The reader's practical task is to identify where a curation chain broke instead of treating every disappointing recommendation as one undifferentiated failure of “AI.”

That distinction changes how performance should be read. A system can retrieve the correct brand and product type while ignoring a budget or compatibility requirement. It can produce an articulate explanation for a link that is invalid. It can recognize that a product is too expensive and still stop searching. These outcomes may look similar in a final answer, but they call for different remedies and carry different levels of confidence.

The benchmark separates search from personalization

AgenticShop's results make the first distinction visible. The benchmark reports a personalization measure based on how well curated products satisfy individual checklist criteria, not simply whether a product belongs to the requested category. Search-augmented systems generally reach only the low-to-mid 30-percent range across many scenarios, while performance is especially weak in Open Exploration. Autonomous web agents also remain modest overall. These numbers are not a forecast for every system; they are results under the benchmark's models, prompts, domains, and evaluation method. Their value is diagnostic: finding a plausible item is much easier than satisfying a particular shopper across several dimensions.

The gap appears clearly when the benchmark measures search ability separately. A product counts as located if it matches the target brand and type, regardless of whether it satisfies the user's checklist. Claude Sonnet 4 Search, for example, reaches a reported 86% hit ratio for Explicit Title queries, while its corresponding personalization score is about 38%. The two measures answer different questions. The first asks, “Did the system reach the right neighborhood?” The second asks, “Did it select and justify an option that works for this person?” A high answer to the first cannot substitute for the second.

The difference is easiest to understand in a concrete case. Suppose a person asks for a replacement battery for an older laptop. A search system may locate the correct manufacturer family and a plausible model. Personalization still requires checking the exact part number, voltage, capacity, safety features, price, warranty, return policy, and seller reliability. If the page lacks one of those facts, the agent may not be able to establish fit. If the agent sees the facts but overlooks the part number, the problem is judgment. If it cites an invalid product page, the problem is attribution. Calling all three outcomes “poor search” would conceal the intervention that each requires.

Query specificity does not remove the gap. In Target Finding, AgenticShop compares exact-title, attribute-specific, and brand-categorical queries. The reported differences are generally marginal. Search-augmented systems may repeatedly rewrite a query, but the rewrites can become overly complex or inconsistent and may overemphasize a salient attribute such as brand.