H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
Zhang et al.
Levels of latent predictors each forecast further ahead and hand subgoals downwards. A three-level hierarchy raises Visual AntMaze success from 18% to 73% with less planner compute.
Source published · In the Daily Brief of 12 October 2026
Filed under Agents, Large language models
The paper asks whether world models that predict in a learned latent space can plan over long horizons if the prediction is organised in levels. Each level is an action-conditioned joint-embedding predictive model that looks further ahead than the one beneath it.
Planning is top-down: the upper level proposes a latent target, and the lower level plans toward it. In four simulated environments the hierarchy outperforms a flat model, and on Visual AntMaze the three-level version reaches 73% success against 18%, using less planning compute.
The authors also report an extension to real-robot videos from DROID, using inverse-dynamics supervision, with better offline planning fidelity at lower compute. Code and weights were released.