VIScore

Does a better latent distribution give better planning?

End-to-end latent world models need an explicit mechanism against collapse, and the usual answer is to regularize the embedding towards an isotropic Gaussian. Whether that property buys planning success has not been tested, so we test it with a controlled substitution: keep the architecture, the data and the predictor loss fixed, and replace only the regularizer.

SIGReg matches random one-dimensional projections of the embedding to a standard Gaussian through an Epps–Pulley statistic. VISReg targets the same prior but factorizes it into independently weighted center, scale and shape terms, which separates two axes: how the constraint is decomposed, and how accurately the distribution is actually matched.

\[\mathcal{L}_{\mathrm{center}}=\tfrac{1}{D}\|\mu\|_2^2,\qquad \mathcal{L}_{\mathrm{scale}}=\tfrac{1}{D}\sum_j\left(1-\sigma_j\right)^2,\qquad \mathcal{L}_{\mathrm{shape}}=\tfrac{1}{KB}\sum_{k}\left\|\operatorname{sort}(\widetilde{Z}w_k)-\mathbf{q}_{\mathcal{N}}\right\|_2^2\]

The two axes come apart. Component flexibility is what helps self-supervised learning — reweighting center/scale/shape to \(\{0.5,0.5,2\}\) lifts ImageNet-LT linear-probe accuracy from \(32.0\) to \(35.1\), most of it on the rare classes. It buys nothing for planning. What does move planning is the second axis: a larger batch, which makes the shape term a finer approximation of the target, raises out-of-distribution success from \(77.4\) to \(82.0\) points while in-distribution success stays flat.

Two things follow. A latent-space property that is demonstrably good in one setting does not transfer to planning by itself, so planning quality has to be measured on the planning pipeline — which is what the rest of this page is about. And single-seed evaluation is not enough to see any of this: under one seed VIS-WM appears to beat LeWM outright, and under three fresh seeds the two are indistinguishable in distribution.

Where the advantage does and does not hold. On six unseen object shapes the apparent gain survives only at the shortest horizon, and the matched-batch control above traces it to batch size acting as a hidden regularization-strength knob for VISReg rather than to the objective itself. Under the conventional single-seed protocol VIS-WM looks ahead of LeWorldModel; under three fresh seeds the two objectives become indistinguishable — a finding about the protocol rather than about either objective.

Why not just run the planner?

Selecting a latent world model means training it, then running model-predictive control for 50 episodes and reading a success rate. That is expensive, it is noisy at a single evaluation seed, and it answers only one question: whether this checkpoint plans well. It never says which part is broken — the encoder, the predictor, or the interaction between the predictor and the search.

The diagnostics in common use do not fill the gap. Straightness of latent trajectories changes sign across tasks (\(+0.51/+0.13/-0.67\) on PushT/Reacher/Two-Room), so it cannot be read as quality. A physical-state probe measures perception, which is necessary but far from sufficient. Latent empowerment is strongly associated with success within a task, but it is measured in nats whose meaning does not transfer: pooled across tasks it collapses, and on converged checkpoints it actively mis-orders them.

The problem with current metrics

A single planning attempt passes through four stages, and it fails if any one of them fails: the vision encoder turns pixels into \(z_t\); the predictor rolls \(z\) forward under a candidate action sequence; the planner searches that space for a sequence whose imagined outcome looks good; and the task tolerance grades whether the state actually reached counts as success. A diagnostic that measures a subset cannot tell a healthy model from one that will fail, because the stage it does not look at is exactly where the failure may sit.

The two uncovered stages are not details. A predictor accurate to a fixed latent distance is harmless for coarse navigation and fatal for precise manipulation — that is the tolerance stage. And a latent landscape can be informative yet still contain optima that the search will prefer over the action that actually succeeds — that is the planner stage. Both are measurable, and measuring them is what separates VIScore from the three diagnostics above.

What is missing is a reading that is anchored to the task. The same absolute latent error is harmless for coarse navigation and fatal for manipulation, so a diagnostic has to be expressed relative to the ball the environment’s own success criterion grades on.

VIScore: Veracity × Influence × Sobriety

Predictor-based planning needs three conditions to hold at once: the predictor must roll the latent state forward accurately enough for the task; different actions must produce sufficiently distinguishable futures; and the planner must not be able to fabricate candidates it merely believes are better than the data’s own actions. We call these veracity, influence and sobriety, and multiply them because they are prerequisites — the weakest unmet one dominates:

\[\mathrm{VIS} \;=\; \underbrace{\operatorname{erf}\!\left(\frac{d_{\mathrm{tol}}/2}{\sqrt{2}\,\sigma_{\mathrm{roll}}}\right)}_{\text{veracity}} \;\cdot\; \underbrace{\min\!\left(\frac{m_{\mathrm{emp}}}{\tau},\,1\right)}_{\text{influence}} \;\cdot\; \underbrace{(1-\hat p)}_{\text{sobriety}}\]

Veracity — does the rollout stay inside the tolerance?

\(\sigma_{\mathrm{roll}}\) is the RMS terminal error of an \(H\)-step open-loop rollout, the regime a planner actually rolls out in, and \(d_{\mathrm{tol}}\) is the latent image of the environment’s success tolerance — measured on probe pairs whose graded physical displacement already equals one tolerance, not extrapolated from a local slope. Only the ratio enters, so rescaling the latent leaves veracity unchanged and models of different latent dimension remain comparable.

Influence — do actions command distinguishable futures, enough for this task?

\[m_{\mathrm{emp}} = \tfrac12\log\det\!\left(I + \hat E_H^{-1}\hat S\right)\] with \(\hat S\) the covariance of \(H\)-step terminal displacements induced by action perturbations and \(\hat E_H\) the predictor’s own residual covariance rescaled to the error the rollout accumulates. This is the Gaussian channel capacity from actions to futures, invariant to any invertible linear reparameterization of the latent. It is then capped at \(\tau=82\) nats, because capacity stops buying success past a knee:

Sobriety — can a search exploit the landscape?

At each of \(K=64\) expert anchors the goal is the future the recorded action reached, so the expert’s own action is a known-good plan. A mini-CEM searches for a lower imagined cost, and sobriety counts how often it wins: \[\hat p=\frac{1}{K}\sum_{k=1}^{K}\mathbb{1}\!\left[J(a^*_k) > J(a^{\mathrm{cem}}_k)\right], \qquad \mathrm{sobriety}=1-\hat p\] Every anchor the search wins is an exploitable hole: the planner will be steered by a hallucinated improvement. Only the sign of each gap enters, which is what makes the probe cheap — \(64\times512\) rollouts, no environment.

from viscore import score_checkpoint  # pip install -e .

f = score_checkpoint("run/lewm_epoch_7_object.ckpt",
                     "probes/probe_pusht.npz", task="pusht")

# VIS 0.7213
# veracity  0.9014   d_tol 4.37   sigma_roll 1.79
# influence 1.0000   m_emp 147.3 / tau 82 nats   [saturated]
# sobriety  0.8003   p_hat 0.200

Scoring a world model from another codebase needs three methods — encode, action_embed, predict_next — and nothing else; adding a task means declaring its success tolerance, read off the environment source rather than tuned against success labels.

Results

Association with planning success

Within each task VIScore is comparable to raw empowerment and ahead of both representation diagnostics. The two instruments separate on the cross-task axis: pooled correlation and cross-task calibration error, where a metric measured in task-specific units cannot transfer. On the pre-registered terminal set of converged checkpoints, raw empowerment’s pooled correlation is negative — worse than using no metric at all — while VIScore predicts a held-out task’s success to within 4 points.

Which factor binds, and where

The product is the summary; the factors are the diagnosis. No single factor dominates every distribution shift, and the strongest one changes with the regime — so a metric that reports only its product hides the part a practitioner needs.

Not an artifact of CEM

Sobriety contains a search by construction, which raises the question of whether the score is specific to CEM. Keeping the checkpoint, goals, episode count and planning cost fixed and replacing only the optimizer, VIScore keeps positive within-task association in all 18 planner × task cells (\(+0.29\) to \(+0.91\), median \(+0.67\)), and it does not decay as the planner weakens: the rank correlation between a planner’s success deficit against CEM and its within-task \(\rho\) is \(+0.07\). Predictive sampling scores \(43.3\) on PushT against CEM’s \(84.7\), and VIScore still ranks that task’s checkpoints at \(+0.76\) — a planner that finds much worse plans still finds better plans with a better model.

What the score does not claim

These are measured boundaries, not caveats added for safety:

• Search-based planning only. An amortized policy — goal-conditioned inverse dynamics, behaviour cloning — never runs a search, so sobriety has no estimand, and its success was measured to be independent of rollout fidelity. The released code can score any subset of the factors, but a partial score is a description of the model, not a validated predictor of that policy’s success.
• Discrete-mode success is out of scope. Cube succeeds only if the gripper enters a closed mode on the block, a discrete transition none of the three continuous factors represents; its influence factor reads exactly \(1\) on every checkpoint. We report it as a missing axis, not a verdict.

Conclusion

VIScore shows that latent world-model quality for planning decomposes into three measurable prerequisites, and that anchoring each of them to the task’s own success tolerance is what makes readings transfer where quantities in nats do not. Because it costs seconds rather than GPU-hours and names the binding factor rather than only ranking checkpoints, it is usable both for model selection and for deciding what to fix — and its factor decomposition is what let us find that a reported difference between two regularizers was a batch-size knob rather than the objective.

Everything needed to check this is released: the code, the checkpoints organized by evaluation pool, and the derived quantities that let the paper’s tables be recomputed on a CPU in seconds, with no GPU, dataset or checkpoint download.