Why Agent Models Need a Data System
This series examines Salesforce AI Research's xLAM family of large action models and its relationship with APIGen, the pipeline used to generate and verify function-calling data. The opening question is simple: why can a language model that writes convincing prose still choose the wrong tool, invent an argument, or make a call that does not answer the user's request? Understanding the answer gives the reader a practical way to judge agent models: not by parameter count or a single leaderboard position, but by how their data is represented, varied, checked, trained on, and evaluated.
When fluent language must become an action
A conversational model can be judged largely by the text it produces. A function-calling model has a more demanding job. It must translate a request into an external action whose name, arguments, types, and timing fit a tool description. If someone asks for the weather in two cities, a useful model must recognize that the request contains two targets, select the weather function, provide the right locations, and return calls in the form the receiving system expects. The call is not the answer by itself; it is a proposal for an operation whose result must still make sense in the context of the request.
That extra interface creates failure modes that ordinary text evaluation can hide. A model may name a function that was never offered. It may choose a real function but pass an argument with the wrong type. It may invent a value that was not present in the request or in an earlier observation. It may produce a syntactically valid call that retrieves something unrelated to the user's goal. An agent dataset can also teach these mistakes if it contains unverified outputs, narrow examples, or interaction records stored in incompatible formats.
The xLAM project treats this as a data problem as well as a model problem. Its report describes an open-source agent-model landscape constrained by scarce high-quality data, heterogeneous formats, limited task diversity, and noisy trajectories. These constraints make it difficult to compare examples, augment them consistently, or transfer what a model learned from one environment to another. The obstacle is not simply that a model needs more facts. It needs repeated, varied examples of the relationship between intent, available actions, arguments, feedback, and the next decision.
This is where the relationship between xLAM and APIGen matters. xLAM is the model family: it includes general agent models and smaller models specialized for function calling. APIGen is an upstream data-generation system. It samples executable APIs, examples, and generation prompts; produces candidate query-and-call pairs; and filters them through format, execution, and semantic checks. The two names therefore describe different layers of the same case study. APIGen helps construct training signal; xLAM is the set of models whose capabilities are tested after that signal is combined with other data and training methods.
The unseen-tool problem makes this relationship practical. A model trained mostly on restaurant-booking examples may learn the surface pattern of making a reservation yet struggle when asked to retrieve stock information. The issue is not only missing vocabulary. The model must recognize a new tool description, map the request to unfamiliar parameters, and respect the same structural rules in a different domain. APIGen's use of many executable APIs is meant to create more such variation, while xLAM's unified format gives those examples a shared shape. This does not guarantee transfer to every new API, but it clarifies what transfer requires: learning how to read an action space and ground arguments, not memorizing a short list of familiar function names.
The open-source significance follows from the same distinction. If only the final checkpoint is visible, a strong result says little about whether the model learned from verified trajectories, noisy synthetic labels, or a narrow benchmark-shaped dataset. The xLAM report presents the processing and synthesis pipeline as part of the contribution, so others can inspect the assumptions behind the model's behavior and potentially adapt the method to their own tool environments. That openness does not make the evidence neutral or complete; it makes the route from data choices to capability easier to question.
The reports do not establish that data quality always matters more than model size, or that xLAM will behave reliably in every deployment. They do provide a concrete engineering argument: if the training data makes the important relationships explicit and tests them against executable tools, a model can learn more than a collection of plausible-looking answers would teach it.
xLAM is a model family; APIGen is the data engine
The most useful mental model is a loop rather than a model checkpoint in isolation. Existing agent interactions and API libraries enter the loop. They are converted into a common representation, varied to reduce dependence on one surface form, checked for structural and semantic problems, mixed into training data, and evaluated on several kinds of agent task. Evaluation then supplies diagnostic feedback about where the data or training recipe needs improvement.
The phrase “data system” names several responsibilities that are easy to overlook. Someone must decide what counts as an available tool and how its parameters are described. Someone must preserve the connection between a user's wording and the call made in response. Someone must decide whether an error is a malformed record, a failed operation, or a semantically wrong answer. Finally, someone must choose evaluations that expose the intended behavior rather than only the format used to express it. In xLAM and APIGen, these responsibilities are distributed across representation design, samplers, checkers, training mixtures, and benchmarks. The benefit of seeing them together is that a failure can be traced to the layer that owns it instead of being blamed vaguely on “the model.”
This loop explains why the reports emphasize a unified representation. The record is organized around task instructions, available tools, output-format instructions, the user's query, and interaction steps. A step can hold a model's plan, its tool calls, the environment's feedback, and a user follow-up. A purely conversational turn can remain in the same structure without pretending that an API was called. Once these pieces have stable roles, the pipeline can inspect whether a call belongs to the available action space and whether the arguments agree with the schema and the preceding context.
Standardization also changes what “more data” means. Without a shared structure, a new dataset may add volume but preserve differences that make examples hard to compare. With one, the pipeline can reorder tool descriptions, vary how sections are joined, rephrase instructions, and test several output formats while keeping the underlying task relationship constant. The representation does not solve every problem: it cannot by itself determine whether a call's value is appropriate to an ambiguous request, and it may not preserve every environment-specific meaning when different sources are converted into the same form. But it creates a place where those questions can be asked consistently.
APIGen extends the same idea to new synthetic examples. Rather than asking a generator to produce free-form demonstrations and accepting whatever sounds reasonable, it gives the generator documented functions and structured objectives. The resulting calls can be checked against the functions and, where the backend is available, actually run. This is a crucial distinction. A textual reviewer might judge get_weather with a location argument plausible; an execution checker can reveal a wrong type, missing parameter, invalid endpoint, timeout, or runtime error. A later semantic check is still necessary because a call can run successfully and return a result that does not address the user's request.
The pipeline therefore separates three questions that are often collapsed into one word—correctness. Is the record well formed? Does the proposed operation execute? Does the operation and its result fit the user's intent? Those questions support different kinds of evidence and different failure handling. The first can often be checked with rules. The second needs an executable backend. The third requires interpreting the request, the available tools, the calls, and the results together, which APIGen handles with another language model as a semantic checker.
What this case changes about “capability”
Seen this way, xLAM is not presented as a magic model that suddenly understands tools. It is a case in which the training environment is designed to expose the constraints of tool use. APIGen's examples include simple calls, choosing among several functions, parallel calls, and combinations of parallel and multiple selection. xLAM's broader pipeline also includes cleaned agent data, augmented records, general instruction data, and preference alignment. The models range from compact function-calling variants to much larger general models, so the reports can ask not only whether the largest system performs well, but whether a disciplined data recipe helps smaller systems compete in selected settings.
That makes the central claim more precise. The evidence supports the idea that data representation, diversity, executable verification, and training mixture are capability levers. It does not isolate APIGen as the sole cause of every xLAM result. The reports do not publish a complete component-by-component attribution for all mixture ratios, base models, augmentation operations, and alignment choices. A strong score is therefore evidence about a combined system, not a clean measurement of one module.
The same restraint applies to evaluation. The xLAM report uses WebShop, ToolQuery, ToolBench, and the Berkeley Function-Calling Leaderboard, each exposing a different layer of behavior. An abstract-syntax check can show that a call has the expected structure; executing the call tests operational validity; relevance detection asks whether the available tools can address the request at all. Web interaction and multi-turn benchmarks add different demands. These measures should not be treated as interchangeable proof of general autonomy.
A score can improve because a model learned the underlying decision, because it became better at a required serialization, or because the test distribution resembles its training examples. The reports try to separate some of these effects by evaluating both structured formats and executed calls, but no benchmark removes the need to inspect the task and interface it represents. This is why the central question is not simply whether xLAM scores highly. It is which part of tool use the score makes visible, and which parts remain outside the test.
There are also time and coverage boundaries. The xLAM report uses a later BFCL v2 snapshot with live user data released after training, while the APIGen report uses an earlier snapshot. One xLAM ToolBench result was unavailable because the evaluation server was down. APIGen's stated implementation covers REST APIs and Python functions and focuses on single-turn function-calling generation. Its authors identify multi-turn interactions, additional programming languages, and more complex agent-tool settings as future extensions. Executable verification is powerful, but it depends on APIs that can be documented, reached, and run; a curated executable library is not the same as the full world of tools a deployed agent may encounter.
Those boundaries are not footnotes to the subject. They define what the data system can legitimately teach. xLAM and APIGen show a route toward more reliable function calling by making the path from intent to action inspectable and testable. The next question is therefore concrete: what must a common representation preserve so that a request, the tools available to answer it, the model's action, and the environment's response remain connected? Answering that question begins with the unified trajectory rather than with another leaderboard score.