SIM
PHOTOREAL
Gravity invertedtraj corr 0.9999
Gravity points up: g = +9.81 m/s². The released ball falls to the ceiling.
InfiniteWorld — Parallel-Physics Worlds
A VideoQA benchmark for physical-law understanding. Videos come from worlds whose laws are systematically altered — gravity inverted, friction removed, collisions super-elastic — each with exact per-frame ground truth from the engine that produced it.
aMethod
Three layers: engine truth → photoreal translation → tracking-based certification.
LAYER 1
Engine truth
PyBullet at 240 Hz with a dialed-in law. Exact trajectory, camera, and depth per frame.
R² ≥ 0.999999 fit
→
LAYER 2
Neural sim→real translation
Depth-conditioned Cosmos-Transfer1-7B re-renders the sim photorealistically, motion locked to the engine.
~953 s / clip on A100
→
LAYER 3
SAM2 coherence certification
SAM2 tracks the translated object against the projected engine trajectory. Fail → discard.
gate: tracked ≥ 0.80 · corr > 0.95
bCertified sim ↔ real pairs
Left: engine sim. Right: Cosmos translation. Every pair passed trajectory certification and a human artifact screen.
SIM
PHOTOREAL
Restitution supertraj corr 0.9980
Super-elastic bounces: e = 1.20 (clamp 8 m/s). Each apex climbs higher than the last.
SIM
PHOTOREAL
Combo: gravity inverted × air dragtraj corr 1.0000
Two laws at once: g = +9.81 m/s² and drag k = 1.2 /s. The resting ball lifts off and decelerates as it climbs.
SIM
PHOTOREAL
Friction scaledtraj corr 1.0000
Friction removed: μ = 0 (Earth ref μ = 0.35). The pushed box slides without stopping.
SIM
PHOTOREAL
Dissipation airtraj corr 1.0000
Thick air drag: k = 0.4 /s velocity decay. The flight arc dies mid-air.
SIM
PHOTOREAL
Gravity scaledtraj corr 0.9993
Gravity at 10%: g = −0.981 m/s². Slow, floaty drop with low bounces.
Physics-preserved = SAM2-tracked trajectory matches the projected engine ground truth. The full 1,200-clip photoreal tier is translating and certifying continuously.
cEngine tier: every law is a dial
Weakest (m0) vs strongest (m4) magnitude per family, plus combos and normal-physics controls. Parameter-regression and law-induction tasks live on these dials.
1,200 anomaly (6 families × 5 magnitudes × 40 scenes) + 1,200 matched controls + 600 combos.
dBy the numbers
3000
simulated clips
(1200 anomaly · 1200 control · 600 combo)
(1200 anomaly · 1200 control · 600 combo)
≥0.999999
min engine-fit R² across all 3000
15,910
QA items · 5 task types
PASS
blind-LLM gate
(gpt-5-mini + deepseek-v3)
(gpt-5-mini + deepseek-v3)
1,200
photoreal tier in flight
(4×A100, rolling SAM2 certification)
(4×A100, rolling SAM2 certification)
1e-9
bit-level reproducibility
(corpus regenerated & verified)
(corpus regenerated & verified)
eThe five QA tasks
Unanswerable without watching the video — enforced by the blind-LLM audit below. Examples from the training split.
law_identification · 3,600 items
In this video, which one of the following aspects of physics behaves differently from normal Earth physics?
- The direction of gravity
- Bounciness (restitution) of collisions
- None — the physics matches normal Earth physics
- The strength of gravity
Show answer
The direction of gravity
parameter_regression · 3,600 items
Estimate the effective vertical gravitational acceleration acting on the object in this video, in m/s². Use a signed value: negative means directed downward, positive means upward.
free-response numeric · unit m/s² · graded within ± toleranceShow answer
−0.981 m/s² (full credit ±0.6, half credit ±1.2)
forward_prediction · 1,540 items
Consider the position of the moving object at t = 1.38 s in the video. At t = 2.50 s, where is the object relative to that position?
- Noticeably lower
- At about the same height
- It is out of view (off-screen)
- Noticeably higher
Show answer
Noticeably lower
cross_world_counterfactual · 1,170 items
If this exact scene were re-run with normal Earth physics (same object, same initial push), would the object slide a shorter total distance, a longer total distance, or about the same distance as it does in the video?
- A longer distance than in the video
- About the same distance
- A shorter distance than in the video
Show answer
About the same distance
law_induction · 6,000 items
The first two videos come from the same world, whose physics differs from normal Earth physics in a specific, consistent way. Does the third video come from a world with the same physics — the same law(s) altered in the same direction and by the same amount?
- No
- Yes
Show answer
Yes
fBlind-LLM gate
Text-only models see question + options, no video (temperature 0, 300 stratified items). A task passes only if both stay below chance + 8 pts. All four types pass.
| Task type | chance | gate | gpt-5-mini | deepseek-v3 | verdict |
|---|---|---|---|---|---|
| law_identification | 25.0 | 33.0 | 18.3 | 25.0 | PASS |
| forward_prediction | 25.0 | 33.0 | 25.0 | 18.8 | PASS |
| cross_world_counterfactual | 33.3 | 41.3 | 34.0 | 37.0 | PASS |
| law_induction | 50.0 | 58.0 | 51.7 | 51.7 | PASS |
Accuracies in %. parameter_regression is free-response numeric and is not part of the MCQ blind gate.
Every released row carries blind_answerable = false.
gLaw families & dials
Six families × five magnitudes, plus matched Earth-physics controls.
| Family | Altered law | dial | C-tier certification |
|---|---|---|---|
| gravity_inverted | Direction of gravity flips upward | g = +9.81 m/s² | pilot 11/12 |
| gravity_scaled | Strength of gravity scaled down | g_scale = 0.1–1.0 | run in progress · 11/12 of certified so far |
| friction_scaled | Surface friction scaled to zero | μ = 0 (ref 0.35) | run in progress · 12/12 of certified so far |
| restitution_super | Super-elastic collisions | e = 1.3 (clamp 8 m/s) | pilot 11/12 |
| dissipation_air | Strong in-flight air drag | k = 0.4 /s | run in progress · 10/12 of certified so far |
| buoyancy_inverted | Inverted buoyancy in liquid | F = 1.8·m·g up | C-tier coming in v1.1 — water-occlusion re-render |
Corpus generation is complete for all 3000 clips (six families × five magnitudes + controls, all fit at R² ≥ 0.999999). C-tier certification of the full corpus is a live run: 91 / 1200 translated clips certified so far, 62 passing. Per-family "of certified so far" counts are from that in-progress run.