Overview

xLAM's results reflect a mixture of cleaned and augmented agent data, synthetic function-calling examples, general instruction data, supervised tuning, and preference alignment—not APIGen alone. WebShop, ToolQuery, ToolBench, and BFCL test different layers, from interactive completion and unseen tools to AST structure, execution, and relevance. The reported scores include strong compact-model function calling, but model variants are not controlled by parameter count, ablations use composite data, and benchmark dates, modes, and missing results limit attribution and autonomy claims.

Learning outcomes

xLAM's final checkpoint combines synthetic calls with agent, instruction, augmentation, base-model, and alignment choices, so gains cannot be assigned to APIGen alone.
AST, executable, relevance, web, and multi-turn metrics expose different capabilities and should not be treated as interchangeable.
In the cited BFCL v2 snapshot, xLAM-8x22b-r scored 87.31% and xLAM-1b-fc-r 75.43%; this supports targeted strength, not controlled scale causality or general autonomy.

Keywords

xLAMFunction-callingBFCLToolBenchToolQueryWebShop

Reading the Results Without Overclaiming

By the time xLAM reaches evaluation, several different kinds of data have already been mixed into the model's training. That makes the results useful evidence about a data system, but difficult to attribute to one component. This chapter explains what enters xLAM training, what each benchmark layer actually tests, and what the reported small-model and ablation results support. The aim is to read the numbers as evidence about specific capabilities—not as a single verdict on general autonomy.

The training mixture is part of the model

APIGen matters to xLAM because it supplies a large, structured source of function-calling examples, but it is not the whole training story. For the general xLAM models, the report combines cleaned and augmented agent datasets, synthetic function-calling data, and general instruction-tuning data. The general instruction portion—drawn from DialogStudio and Data Provenance sources after filtering and deduplication—makes up 20% to 30% of the training set. The authors preserve the original formats of that general instruction data to support robustness rather than converting everything into one agent-specific form.

The specialized function-calling variants use a more targeted mixture. For xLAM-7b-fc-r and xLAM-1b-fc-r, 50% of the training data comes from the high-quality synthetic function-calling set, with the remaining half sampled from other tasks. These models are trained for simple, multiple, parallel, and parallel-multiple calls, as well as relevance detection. The mixture therefore expresses a design choice: synthetic calls receive enough weight to shape tool behavior, while other data remains available for broader instruction following.

Training then applies supervised fine-tuning followed by preference alignment through Direct Preference Optimization. The report describes full fine-tuning for most supervised training, low-rank adaptation for the largest model and for preference alignment, and distributed training across accelerators. These details matter less as an inventory than as a reminder that the final checkpoint is the product of several stages. A benchmark result cannot be assigned to APIGen simply because APIGen supplied an important dataset. Base-model capabilities, cleaned trajectories, prompt augmentation, general instruction data, and alignment all contribute.

The model family makes this attribution question visible. It includes general models at different scales and specialized function-calling models based on smaller code-oriented systems. Comparing them can reveal whether a compact model has learned useful tool behavior, but it does not create a perfectly controlled experiment. The models can differ in base model, context length, training mixture, and target category as well as parameter count. “Smaller model” is therefore an important description, not a complete causal explanation.

A sensible interpretation starts by asking what the training mixture was designed to teach. APIGen-like data should help with structured calls, argument grounding, and multiple-call patterns. General instruction data can help preserve broader language behavior. Cleaned and augmented agent data can expose trajectories and varied prompt formats.

Published by DeepMox

FAQ

What does BFCL's AST evaluation measure?

It checks whether generated calls match documented function structure, including names, required parameters, and types; it does not establish execution or relevance.

Why are xLAM's 1B and 7B results not a pure size comparison?

They use different base models, context lengths, training mixtures, and targets, so parameter count is not the only changing factor.

What does a missing xLAM ToolBench result imply?

The xLAM-8x22b-r comparison is incomplete because the ToolBench server was down; its unreported performance cannot be inferred.

Previous11 minAPIGen's Diverse Training Data

Next12 minxLAM and APIGen: System Reliability

xLAM Results Without Overclaiming

Reading the Results Without Overclaiming

The training mixture is part of the model

FAQ