إنتقل إلى المحتوى الرئيسي

World Models: What You Can Actually Build Today

متوسط

On September 28, 2026 AMD announced it is acquiring World Labs for about $8.2 billion in stock — its second-largest deal ever, behind only Xilinx — and Fei-Fei Li will join as executive vice president and chief scientist reporting to Lisa Su. A day later OpenAI's DevDay took the front page. Between them, "world model" became the phrase of the week.

Here is the problem with the phrase. Two products that share it are architecturally opposite. One keeps the world inside the model and streams you frames as you move; it is interactive, it cannot be exported, and a session is capped in seconds. The other generates a 3D scene you download, own and render yourself for free; it is persistent, engine-ready, and completely inert. Picking the wrong one is not a preference mistake, it is a re-architecture.

This page is the practical split: what each family really produces, the verified API and export details, the one benchmark that settles a common assumption, the open weights you can run on your own GPU, and the five things neither family will do for you.

What you'll learn
  • Tell the two families of world model apart, and pick by what you need to persist
  • Know the real limits: the 60-second wall, the physics disclaimer, the export tier that gates commercial rights
  • Read the World API correctly — async operations, the WLT-Api-Key header, and which splat formats come out
  • Use the WorldScore evidence instead of vibes when choosing between a video model and a 3D model
  • Run an open world model yourself, and understand why inverse dynamics is the part worth $8.2 billion

Two things share the name​

Frame generators3D generators
ExamplesGenie 3 / Project Genie, driving world modelsMarble (World Labs)
What it outputsA video stream, frame by frame, conditioned on your actionsA scene: Gaussian splats, panoramas, optionally a mesh
Where the world livesInside the model's rolling contextIn a file you download
Interactive?Yes — that is the whole pointNo. You navigate it, but it does not react
Exportable?No geometry to exportYes — SPZ, PLY, GLB, collider meshes
Cost shapePer second of interaction, per viewer, foreverOnce at generation. Rendering is then yours
Fails byDrifting, forgetting, running out of sessionBeing static and not simulating anything

If you need a world that responds — a player pressing a key, a robot policy taking an action and seeing what happens — you need a frame generator. If you need a world that persists — an asset in your game, a set in your render pipeline, a scene five users load next month and see identically — you need a 3D generator. Almost every confused "why can't it just…" question on this topic is someone holding one and asking for the other.

The frame generators, and the 60-second wall​

Google's Project Genie went from research preview to a public-ish product on January 29, 2026. The access shape is narrow and worth knowing before you plan around it: Google AI Ultra subscribers, United States only, 18+, and explicitly not available on Google AI Ultra for Business. Google calls it an "experimental research prototype".

The limit that teaches you the most is Google's own: generations are limited to 60 seconds. That is not a billing decision, it is a state problem. In a frame generator the world is not stored anywhere — it is implied by the model's context. Every frame is predicted from the prompt, the accumulated frames and your actions. Consistency is therefore bounded by the same thing that bounds a long chat: how much history the model can hold and keep coherent. Walk away from a wall and back, and whether the wall is still there is a memory question, not a rendering one.

Google's stated caveats are the rest of the honest spec sheet: generated worlds "might not look completely true-to-life or always adhere closely to prompts or images, or real-world physics", and characters "can sometimes be less controllable, or experience higher latency in control".

Read that physics line carefully if you were planning to use a frame generator as a simulator. It is a plausibility engine, not a solver. Nothing guarantees momentum is conserved between frame 300 and frame 301.

The frame budget nobody mentions​

Interactive generation at the commonly reported 24 fps gives you about 41.7 ms per frame, end to end, including your input reaching the server. That single number explains most of the product decisions in this family: why sessions are short, why access is gated to a paid tier, why there is no public API yet, and why the cost curve is per viewer-second rather than per asset. You are not buying a file. You are renting a GPU that must beat a 42-millisecond deadline for as long as someone is looking.

The 3D generators, and what you actually receive​

World Labs' Marble turns text, an image, a panorama, multiple views or a video clip into a navigable 3D scene. The important engineering fact is the representation: the output is 3D Gaussian splats, not a mesh.

Splats are millions of small, oriented, semi-transparent blobs with colour and view-dependent shading. They photo-reconstruct beautifully and render fast on commodity GPUs. They also have no surfaces, no edges and no faces — which is why the next section matters more than it looks.

What Marble's own export docs say you get:

ExportAvailability
Gaussian splats — SPZ and PLYStandard plan
360 panoramasStandard plan
Collider meshesStandard plan
Textured mesh (GLB) + commercial rightsPro plan only
Anything at allNot on Free — generate only

Two gotchas live in that table. First, collider meshes are a separate export — you need them because you cannot raycast, collide or do physics against splats directly. The splats are what the player sees; the collider mesh is what the engine tests against. Ship only the splats and your beautiful world has no floor. Second, commercial rights are tied to the Pro tier, not to the act of exporting. Check the plan before you put a generated world in a product.

Marble's docs also note that lower-resolution exports are "optimized to be as perceptually similar as possible to the higher-resolution files" — a hint that you should test the small tier before paying to ship the big one.

SPZ: the 10× that is not World Labs'​

A detail several write-ups get wrong: SPZ is Niantic's format, open-sourced by Niantic Spatial, not a World Labs invention — Marble simply exports to it. It is worth using anyway. SPZ stores each spherical-harmonics coefficient as an 8-bit signed integer with optional further quantisation, landing at roughly 10× smaller than the equivalent PLY with no perceptible quality loss. Niantic's own testing puts around 5 bits as the sweet spot for most scenes, with essentially-zero MSE at 8 bits and about 18× compression at 3. SPZ 4, released in 2026, splits compression into six parallel ZSTD streams — one per attribute class — for roughly 3–5× faster compression and 1.5–2× faster end-to-end loading at the same ~10× size advantage.

If you are shipping splats over the web, that is the difference between a scene that loads and one that does not.

The API, for real​

The World API exposes Marble's generation to your own code. The verified shape, from World Labs' docs index and its official Python client:

  • Inputs: text, images, panoramas, multi-view inputs, video.
  • Outputs: splats in SPZ, PLY, SOGS, KSPLAT and SPLAT.
  • Auth: an API key passed in the WLT-Api-Key header; the Python client reads WORLDLABS_API_KEY from the environment.
  • Execution model: asynchronous. You submit a generation, get an operation back, and poll until it is done. There is also a credit-balance endpoint, media upload/retrieve, and world create/retrieve/list/export/delete.
  • Client: worldlabsai/worldlabs-api-python, Python 3.12+. A CUDA GPU is only needed for local rendering, not for API use.
  • Pricing: usage-based.

Generate a world from text and poll for it

# github.com/worldlabsai/worldlabs-api-python  (Python 3.12+)
# export WORLDLABS_API_KEY=...

from worldlabs_api.client import WorldLabsClient
from worldlabs_api.models import WorldTextPrompt, WorldsGenerateRequest

client = WorldLabsClient(api_key="your-key-here")

request = WorldsGenerateRequest(
  display_name="Mystical Forest",
  world_prompt=WorldTextPrompt(text_prompt="A mystical forest"),
)

op = client.generate_world(request)      # returns an operation, not a world
done = client.poll_operation(op.operation_id)   # poll until done is true
world = done.response

Design your integration around that async boundary from day one. World generation is a job, not a request — the same lesson as batch inference or video generation. If your handler waits inline for a world, you will rewrite it.

Rendering it without a game engine​

World Labs also maintains Spark, an MIT-licensed 3D Gaussian splatting renderer for THREE.js / WebGL2, so a generated world can ship as a web page. It loads .PLY, .SPZ, .SPLAT, .KSPLAT and .SOG, fuses splats with ordinary meshes in the THREE.js pipeline, and targets the ~98% of devices with WebGL2.

Spark 2.0 (April 15, 2026) is the release that makes big scenes viable in a browser: a level-of-detail splat tree that renders a fixed number of splats per frame regardless of scene size, a streaming .RAD format that supports HTTP Range requests so a coarse proxy appears almost instantly and refines progressively, and virtual splat paging against a fixed GPU memory pool so scenes larger than GPU memory still render — including on phones and Quest 3.

Put a generated world on the web

npm install @sparkjsdev/spark three
# docs and live examples: https://sparkjs.dev
# repo: https://github.com/sparkjsdev/spark
#
# Export from Marble as .spz (≈10x smaller than .ply), serve it as a
# static file, and load it into a THREE.js scene with Spark.
# For scenes over a few hundred MB, convert to .RAD so the viewer can
# range-request a coarse proxy first.

Note what just happened to your cost curve. After generation, you are serving a static file from a CDN and rendering on the user's GPU. Serving the world is free. That is the inverse of the frame-generator model, where every viewer-second is inference.

The evidence: WorldScore​

"Just use a video model" is the most common shortcut here, and there is now a benchmark that answers it. WorldScore (Stanford, arXiv:2504.00983) is a unified benchmark for world generation across 3D, 4D, text-to-video and image-to-video models — about 3,000 examples, with metrics for controllability, consistency, perceptual quality and motion, and explicit 3D consistency and camera-control measures.

The result that should change a decision: on WorldScore-Static, dedicated 3D scene generators lead — WonderWorld at 72.69 and LucidDreamer at 70.40 — clearly above the video models. If your requirement is a geometrically consistent world you can move a camera through, a video model is the wrong tool even when its individual frames look better.

Two caveats. The leaderboard is live and moves as teams submit. And WorldScore-Static measures exactly what its name says — a static world; strong static consistency tells you nothing about dynamics.

Open weights: Cosmos 3, and what AMD actually bought​

You do not need a vendor's API to have a world model. NVIDIA published Cosmos 3 — an open omnimodel for physical AI — with weights on Hugging Face:

ModelSizeRuns on
nvidia/Cosmos3-Edge4BEdge devices, robots, vision agents
nvidia/Cosmos3-Nano16B = 8B reasoner + 8B generatorWorkstation class, e.g. RTX PRO 6000
nvidia/Cosmos3-Super64B = 32B reasoner + 32B generatorHopper / Blackwell, large-scale synthetic data

Architecturally it is a Mixture-of-Transformers handling text, image, video, audio and action in one model, pairing an autoregressive subsequence for reasoning with a diffusion subsequence for generation. It is wired into Hugging Face Diffusers, so the entry point is ordinary Python.

Run an open world model locally

pip install --upgrade diffusers transformers accelerate

# Cosmos3-Nano is 16B (8B reasoner + 8B generator); check the model
# card on Hugging Face for the licence and the exact torch version
# before you build on it.

import torch
from diffusers import Cosmos3OmniPipeline

pipe = Cosmos3OmniPipeline.from_pretrained(
  "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16
)

Now the part that explains the price tag. Cosmos 3's task table is not a list of video tricks:

InputOutputWhat it is
Text / image / videoVideoVideo generation
Text / videoTextVision-language reasoning
Action + image + textVideoForward dynamics — "if I do this, what happens?"
Text + videoActionInverse dynamics — "what actions produced this?"
Image + textVideo and actionPolicy generation

Inverse dynamics is the quiet one. It turns footage into labelled actions — which means ordinary video becomes robot training data, and a generated world becomes a place to train and evaluate a policy without hardware. AMD's own release describes World Labs as building models that "generate, reconstruct and simulate interactive 3D environments" plus technology for "robotic learning and simulation", and frames the deal around reasoning, robotics, simulation and physical AI informing its future silicon roadmap.

Read as a compute story rather than a graphics story, $8.2 billion for a training-environment flywheel is a different kind of bet than $8.2 billion for a text-to-3D tool.

Five jobs a world model will not do for you​

Guided walkthrough1 of 5
  1. None of your chain-of-thought, XML-tag or tool-use technique carries over. Conditioning a world model is closer to directing a camera than to instructing an assistant: what matters is the input image, the viewpoint, the described lighting and the action sequence. The one habit that does transfer is source discipline — a clean, well-framed conditioning image beats an elaborate paragraph.

Choosing, in one pass​

  • Game or product asset, multiple users, must look the same tomorrow → 3D generator. Export SPZ for the web or GLB for an engine, and remember the collider.
  • Interactive exploration, one user, novelty is the point → frame generator, and design inside a 60-second session.
  • Training or evaluating a policy → open weights with forward and inverse dynamics, on your own GPU, with a real simulator for anything safety-relevant.
  • A camera moving through a consistent scene → not a video model. WorldScore says so.
Spatial-intelligence vocabulary
اضغط Enter أو مفتاح المسافة لقلب البطاقة. استخدم مفتاحي السهمين الأيسر والأيمن للتنقل بين البطاقات.تم إظهار المصطلح.
1 / 10

Quiz​

Check yourself

0/5
  1. You need a 3D environment that five different users load next month and see identically. Which family fits?
  2. Why is Project Genie's 60-second generation cap a design consequence rather than a pricing choice?
  3. You export a Marble world as SPZ, drop it into your engine, and the player falls through the ground. What is missing?
  4. WorldScore-Static is led by 3D scene generators (WonderWorld 72.69, LucidDreamer 70.40) rather than video models. What does that justify?
  5. In Cosmos 3's task table, which capability turns ordinary video into robot training data?

Sources & further reading​

Next​