APIGen: From Prompts to Executable Examples
APIGen changes the data problem from “how do we rewrite existing agent records?” to “how do we create new query-and-call pairs that can be tested?” Its design matters because synthetic text is easy to produce and difficult to trust when no one can tell whether the proposed action would work. This chapter follows one candidate through APIGen's lifecycle—from API and seed selection to three verification gates—so the reader can see exactly why execution-grounded data is stronger than an unverified plausible answer, and where that strength ends.
Generation is constrained before the model writes
APIGen does not begin with an empty prompt. It samples one or more APIs from its library, selects example query-answer pairs as seed data, and chooses a prompt template for the generation objective. The API descriptions define the functions and parameters that may appear. The seed examples show the generator what a query and its corresponding calls look like. The prompt template asks for a particular kind of behavior, such as a simple call, several choices among tools, or multiple calls in response to one request.
These inputs are converted into a standardized JSON form before generation. That common form gives the generator a clear set of fields to fill and gives later checkers a predictable object to inspect. APIGen can therefore use the same central logic with different API sources by writing converters for those sources, rather than redesigning the entire pipeline each time a library changes. The uniform structure also makes it possible to reject a candidate whose function or argument does not belong to the supplied API descriptions.
The generator prompt adds variation within those constraints. The examples in the report ask for practical scenarios, plausible parameter values, different difficulty levels, and correctly typed and ranged arguments. A prompt can ask for multiple independent requests in one natural-language query, including cases that require repeated use of one function or use of different functions. APIGen can generate several pairs in one inference by supplying a number of requested examples, which the report describes as a way to reduce token cost. Batching changes how generation is performed, but it does not change the requirement that each resulting pair must pass the same checks.
The seed examples are more than demonstrations that disappear after one generation call. Data points that pass all verification stages are added back to the seed dataset. In principle, this gives later generations more varied references and helps the collection process expand beyond its initial examples. It also creates a dependency: the seed pool can only improve future diversity if the acceptance filter is trustworthy. If weak or semantically wrong examples are allowed back into the pool, the same feedback loop could spread their errors.
A candidate pair contains at least two linked parts: a user query and one or more function calls presented as the answer. The query expresses the user's intent in natural language. The calls select functions and fill arguments. A candidate is useful only if the second part can address the first using the functions that were actually supplied. APIGen's design makes that relationship explicit before a model trained on the data ever sees it.