Transfer Has a Shape, Not a Magic Number

SIMA’s results are most informative when read as a profile of different kinds of transfer rather than as one general-intelligence score. This chapter interprets performance across environments, the specialized and ablated baselines, held-out-world tests, classifier-free guidance, and a focused human comparison. The judgment it supports is deliberately bounded: SIMA shows meaningful language-grounded behavior and positive cross-world transfer, but the evidence does not collapse environment-specific skill, zero-shot reuse, and human-level embodied competence into the same claim.

The multiworld result is an improvement over a local expert

The first result is uneven performance. SIMA performs better in Playhouse and WorldLab, which are comparatively simpler research environments, and less well in the more complex commercial games. Construction Lab is difficult despite being a research setting, because its tasks demand more precise object manipulation and an intuitive grasp of physical relationships. Across skill categories, movement-like behaviors tend to be easier than tasks involving precise spatial understanding, tool use, combat, or building.

That unevenness is not merely a disappointing average. It reveals what kind of grounding the agent has acquired. A policy can often connect language to broad orientation or movement patterns while struggling when success depends on a narrow contact point, a sequence of manipulations, or an environment-specific physical consequence. “Go toward the target” leaves room for approximate control. “Use this tool on that object without touching the distractor” requires the language, visual selection, spatial model, and motor timing to remain aligned. The performance profile exposes those differences.

SIMA’s central comparison is with environment-specialized agents. A specialized agent is trained only on data from one environment, while the main SIMA agent is trained across the portfolio. Performance is normalized relative to the specialist for each environment, making the specialist’s reference level 100 percent in that comparison. On that relative measure, SIMA is reported to achieve a 67 percent average improvement over environment-specialized performance and to outperform the specialist significantly in each of the seven evaluated environments.

The interpretation is positive transfer. Experience from other worlds is not merely harmless extra data; under the reported conditions, it improves the main agent’s average performance even against an agent focused on the target environment. The likely value comes from shared patterns such as navigation, object recognition, and interaction conventions, combined with representations that can reuse experience across visual settings. The result supports the thesis that a multiworld training distribution can help a policy, rather than assuming that specialization is always the strongest route.

The 67 percent figure needs to be read carefully. It is a relative, normalized comparison, not an absolute success rate and not a claim that SIMA is 67 percent closer to human general intelligence. The environment-specialized baseline is a reference for what the same broad method and available data can achieve when confined to one world. The aggregate also averages across environments, while the absolute task difficulty and evaluation method differ. A relative gain says that multiworld training helped under those comparisons; it does not erase the low-performing skills or make the worlds equivalent.