What It Takes to Curate a Product
A shopping agent has done more than retrieve a product only when it can connect that product to a particular person's needs and show why the connection is credible. This chapter examines how AgenticShop turns that demanding idea into a testable pipeline: first by representing different shopping situations and user preferences, then by tracing each recommendation to a real product page and criterion-level evidence. The result is a way to distinguish personalization from a plausible-sounding list.
The distinction matters because “relevant” is often too weak a standard for commerce. A recommendation can belong to the requested category and still fail on price, compatibility, appearance, review quality, shipping, or seller reliability. Conversely, a product may be excellent in the abstract but unsuitable for this person at this time. Curation is the work of resolving those conditions together.
Three shopping intents make the user model testable
AgenticShop begins by treating shopping intent as part of the task, not as incidental wording. Its benchmark separates three situations.
Target Finding starts with a specific item or a fairly precise product description. The user may name an exact title, describe attributes, or combine a brand with a category. The agent's job is not merely to find the name. It must locate the product in an open-web environment, compare where it is sold, and determine whether the available offer satisfies the person's constraints.
Alternative Selection starts with a category rather than a committed item. Someone may be looking for headphones, a household product, or a piece of clothing without knowing which model to choose. The agent must construct a comparison set and explain trade-offs among alternatives. The challenge is partly retrieval, but it is also deciding which differences deserve attention and which options are meaningfully comparable.
Open Exploration removes even the explicit product query. The person asks the system to browse for interesting products based on their profile. The agent must infer a plausible direction, explore across domains, and balance relevance with novelty. It cannot simply match words in a request because the request supplies little product vocabulary. Its search policy becomes part of the recommendation.
These intents form a useful gradient of delegated judgment. Target Finding gives the agent a relatively narrow destination. Alternative Selection delegates more of the work of defining the choice set. Open Exploration delegates both discovery and the first interpretation of what might be worth seeing. The benchmark's taxonomy is not a universal theory of shopping, but it makes a crucial difference visible: the same system may appear capable on a known-item lookup and still fail when it must decide what to search for.
The benchmark also separates product domains because the evidence burden changes with the object. Grocery products often have identifiable titles and attributes. Electronics can demand detailed technical specifications and compatibility checks. Home products may require practical comparisons, while fashion adds subjective visual and stylistic judgments. In open exploration, the agent has to move across these kinds of evidence rather than operate inside one stable catalog. A single overall score would hide these differences.
To make the person behind each task more than a generic prompt, AgenticShop starts from purchase histories and review text. The paper reports 50 user profiles for each of seven scenarios, producing 350 personalized tasks, with roughly 19–20 checklist criteria per profile. The selection process focuses histories on a user's recurring product interest, then generates three linked representations:
- a narrative context describing general preferences, constraints, and decision tendencies;
- a query or shopping situation expressing what the person is trying to do now; and
- a checklist that turns requirements into individual criteria that can be checked against a product.
The three representations are related but not interchangeable. The context gives the agent a broader picture. The query gives the current search direction. The checklist supplies an evaluation instrument. A query may say “find a replacement laptop battery,” while the context reveals a preference for third-party brands, reliable warranty service, and a price below typical original-equipment replacements. The checklist can then ask whether a candidate states the exact part number, voltage, capacity, safety protections, price, warranty, and return process. Each layer narrows a different kind of ambiguity.
The checklist covers six dimensions: brand preferences, price sensitivity, review sensitivity, functional requirements, aesthetic preferences, and purchase preferences. This is not an invitation to count every attribute mechanically. It is a reminder that a shopping decision crosses evidence types. Brand may be a trust signal rather than an absolute rule. Price may be conditional on durability. Reviews may matter only when they discuss the use case. Functional requirements may be hard constraints, while aesthetics may be a preference that can be traded against cost. Shipping and returns can determine whether an otherwise suitable product is practically buyable.
The distinction between hard requirements and soft preferences is especially important. A battery that does not fit the target laptop is not rescued by attractive packaging. A garment that meets the color preference but crowds the wearer may fail the actual purpose. An agent that treats every checklist item as equally decisive can produce brittle results; one that ignores the items can produce generic recommendations. The representation is useful only when it preserves the relationship among criteria and the context in which trade-offs are acceptable.
There is also uncertainty at the point where behavior becomes a profile. A purchase may have been a gift, a review may describe an unusual experience, and repeated purchases may reflect convenience rather than deep brand loyalty. AgenticShop's checklist is therefore an operationalization of preference, not a direct reading of a person's complete utility function. Its value comes from making assumptions inspectable and comparable. It gives the agent and evaluator somewhere specific to disagree.
The whole construction can be viewed as a chain that moves from a person's history to a judgment about a product:
The diagram's important feature is the split between what guides search and what evaluates the result. A query can help an agent find a page without proving that the page satisfies the user's preferences. A context can make a recommendation sound personalized without supplying evidence for its claims. The checklist connects the representation to an observable test.
From a returned link to a defensible judgment
The evaluation begins only after the agent has proposed products. This is a deliberate choice. The system is judged on what it actually returns, not on an imagined product that its explanation seems to describe. A direct link to a specific product page is therefore part of the output contract. A generic search-results page, a category listing, or an invalid URL cannot reliably establish which item the agent selected.
The next step is evidence extraction. AgenticShop renders the product page rather than relying only on the first text response. It collects screenshots of the fully rendered page, parses visible content, and extracts available structured metadata such as product data. This combination matters because online stores are heterogeneous. Some relevant information is loaded dynamically, some is expressed visually, and some is encoded in machine-readable fields. A URL by itself may identify a destination while still revealing very little about the current offer.
The extraction process must also identify the main product. A page may contain variants, recommended alternatives, advertisements, and cross-sell banners. If the system accidentally takes the price of a nearby item or the specifications of a different model, the later evaluation can be internally consistent and still wrong. AgenticShop's extraction instructions therefore emphasize separating the main product from variants and promotional content and collecting information about descriptions, shipping, and reviews attached to that product.
This is a boundary between attribution and explanation. A model can write a detailed paragraph from the user's context even when the underlying link is malformed or leads nowhere. It can also attach a true feature to the wrong product. Evidence-grounded evaluation asks a prior question: is there a specific, accessible object to which the claims refer? Only then does it make sense to ask whether the object fits the person.
Once product information is extracted, an LLM judge evaluates each product against each checklist criterion. The judgment is binary at the item level—satisfied or not satisfied—and includes reasoning tied to the available product data. Missing product data is treated as failure rather than silently filled with a favorable assumption. The evaluation can then aggregate the results, but the individual annotations remain important. They show whether a low score came from brand mismatch, an out-of-budget price, absent reviews, missing technical specifications, or a failure to document shipping and returns.
The design resembles a test suite for a decision, but it should not be mistaken for a complete measure of human satisfaction. Binary criteria simplify trade-offs. A person may accept a slightly higher price for a better warranty, or value one functional feature more than three aesthetic matches. The checklist makes the decision auditable at a useful level of granularity; it does not eliminate the need to decide which criteria matter and how they interact.
AgenticShop performs two forms of reliability check. First, it compares the generated checklist with other representations of the same shopper, including purchase histories, queries, and narrative contexts, using ratings as a ground-truth signal for the study. The checklist shows the strongest reported alignment on average. This supports its use as an evaluation representation, but it does not prove that an LLM has recovered every preference correctly.
Second, the study compares LLM judgments with human judgments. The full approach combines the product information extracted from the page with screenshots and the URL. In the paper's human comparison, this setting reports an overall Cohen's kappa of 0.732 and Spearman's rho of 0.7425. Removing extracted product information, or judging from a URL without the same supporting evidence, produces lower agreement. The practical lesson is not that screenshots or structured extraction guarantee truth. It is that judgment quality depends on giving the evaluator concrete, product-specific material rather than asking it to infer from a link or a polished answer.
The repository implementation makes the ownership boundaries visible. Profile construction produces the inputs for curation; a search prompt asks for a small set of direct product pages and reasons connected to the user's needs; extraction produces structured product and review information; evaluation consumes that information with the checklist. The code is an implementation of the benchmark's intended workflow, not independent evidence that every generated profile or judgment is correct. That distinction matters whenever a pipeline uses one language model to construct a representation and another model-mediated step to score it.
A defensible curation result therefore has at least four linked properties. It identifies a real product rather than a plausible label. It preserves current evidence about that product rather than relying on generic knowledge. It checks the relevant dimensions of the user's need rather than one visible attribute. And it exposes enough reasoning that a person can challenge a mistaken interpretation. If any link is missing, the result may still be useful, but its confidence should fall accordingly.
The pipeline also changes how failure should be interpreted. A product that does not satisfy a checklist item is not automatically evidence of a weak search model; the item may have been unavailable on the web page. A valid product page with poor checklist alignment is a different failure from an invalid link. A well-grounded candidate that is rejected because the agent ignored a hard constraint points to a different problem again. Separating these cases is the beginning of diagnosis.
That separation is what makes the next question possible. If agents can find pages and produce personalized explanations, why do they still miss the best fit? The answer requires looking at the breaks between discovery, attribution, current evidence, comparison, and recovery—not collapsing them into one score for “shopping ability.”