World Models: A Survey of Architectures and Training Methods, and NextWM, a Browser-Native Latent World Model Built on Next.js
https://github.com/gwrxuk/world
Abstract
World models, learned simulators that predict how an environment responds to actions, have become central to model-based reinforcement learning, robotics and interactive video generation. This paper surveys the main families of world models: latent state-space models (World Models, PlaNet, Dreamer), value-equivalent models (MuZero, TD-MPC), joint-embedding predictive architectures (JEPA) and large generative video and interactive-environment models (DIAMOND, GameNGen, Genie, Cosmos). It then explains how they are trained: data sources, objectives, architectures, multi-step training, stabilising techniques, compute and evaluation. We then present NextWM, a 108,577-parameter latent world model implemented in dependency-free TypeScript and served by Next.js. The same code trains in a browser Web Worker and in a server route handler. NextWM learns a pixel-based navigation task from 10,000 exploration transitions, imagines open-loop futures that beat a copy-last-frame baseline from horizon 2 onward, and plans with the cross-entropy method in its latent space. Our first configuration planned no better than random despite low pixel error. An ablation shows why: pixel error barely varied across configurations, while the model’s ability to rank action sequences by return rose from 0.42 to 0.82 (Spearman correlation) when the exploration data covered sustained actions and the reward loss was up-weighted. With one further round of planner-collected data, NextWM reduces the mean final distance to the goal from 0.91 (random) to 0.38 and succeeds in 13% of episodes, against 100% for a planner with the true simulator. The code and every reported number are reproducible with one command per experiment.
Keywords: world models; model-based reinforcement learning; latent dynamics; video generation; planning; cross-entropy method; Next.js; WebAssembly-free in-browser training
1. Introduction
A world model is a learned function that predicts how an environment will change in response to an agent’s actions. Given a history of observations and a proposed action, it returns a distribution over what comes next: the next observation, a compact latent state, a reward, or all three. An agent that has such a model can act by imagining the consequences of its choices instead of only reacting to the present.
The idea is old. Craik [1] argued that organisms carry “small-scale models” of external reality, and Sutton’s Dyna architecture [2] placed a learned model inside a reinforcement-learning (RL) loop. What changed after 2018 is scale and representation. Deep latent-variable models made it possible to learn dynamics directly from pixels [3, 4], and large generative video models made it possible to simulate open-ended, photorealistic scenes that respond to control inputs [13–18].
Today the term covers research lines that rarely cite each other. Model-based RL builds compact latent models (PlaNet, Dreamer, MuZero, TD-MPC) to support planning and policy learning. Generative modelling builds large video and interactive-environment models (Genie, GameNGen, DIAMOND, Cosmos, Sora-class generators) to render plausible futures. A representation-first line, the joint-embedding predictive architecture (JEPA), argues that prediction should happen in an abstract embedding space, not in pixels [20–23].
This paper makes three contributions:
1. A unified survey. We define world models formally, place the main systems in one taxonomy, and compare them on state representation, prediction target and use (Sections 2–3).
2. A practical training guide. We describe how world models are trained: data, objectives, architectures, multi-step training, stabilising techniques, compute and evaluation (Section 4).
3. NextWM. We design and release a small latent world model written entirely in TypeScript, with no machine-learning dependencies, served by Next.js. The same code trains in a browser Web Worker and in a Next.js server route, and it plans with the cross-entropy method inside its own imagination. We report measured results, including a negative result on planning, and the ablation that fixes it (Sections 5–7).
2. Background and Formal Definition
We model the environment as a partially observable Markov decision process (POMDP) with hidden state s_t, action a_t, observation o_t and reward r_t. The agent never observes s_t, so a world model replaces it with a learned latent state z_t that summarises the history. Most world models factor into four learned components:
Encoder: z_t ~ q_φ(z_t | z_{t−1}, a_{t−1}, o_t) (1)
Dynamics: ẑ_{t+1} ~ p_θ(ẑ_{t+1} | z_t, a_t) (2)
Decoder: ô_t ~ p_θ(o_t | z_t) Reward: r̂_t ~ p_θ(r_t | z_t) (3)
The dynamics model (2) is the world model proper; the encoder grounds it in observations. The decoder is optional. MuZero, TD-MPC and JEPA-style models drop it and never reconstruct pixels, while generative video models keep it and make it the main output.
A world model is used in one of three ways:
• Planning: search over action sequences inside the model and execute the best first action (model-predictive control with CEM [26] or MPPI [27], or Monte Carlo tree search [8]).
• Policy learning in imagination: roll the model out to generate synthetic trajectories and train an actor-critic on them [5, 25].
• Simulation: let a person or another agent interact with the model as if it were the environment [13–16].
Two properties separate a world model from a plain video predictor. It must be action-conditioned, so different actions yield different futures, and it must be rollable, so its own predictions can be fed back as inputs for many steps without collapsing.
3. Survey of Existing World Models
Existing world models split into three families by what they predict: compact latent states for control, abstract embeddings for representation learning, or full video frames for simulation. Table 1 places representative systems side by side.
Table 1. Representative world models, grouped by family.
3.1 Latent world models for control
Ha and Schmidhuber [3] trained three parts separately: a variational autoencoder (VAE) [29] that compresses each frame, a mixture-density recurrent network that predicts the next compressed frame, and a small linear controller. The controller was evolved entirely inside the learned model and then transferred to the real CarRacing and VizDoom tasks. This showed that a policy can be trained in a dream, and also that it can exploit the model’s errors.
PlaNet [4] introduced the recurrent state-space model (RSSM), whose state has a deterministic path (a GRU) that carries memory and a stochastic path that captures uncertainty; PlaNet planned with CEM at every step. Dreamer [5] kept the RSSM but replaced online planning with an actor-critic trained on imagined rollouts, backpropagating value gradients through the learned dynamics. DreamerV2 [6] switched to categorical latents and reached human-level Atari performance. DreamerV3 [7] added symlog predictions, free bits, return normalisation and KL balancing, used one set of hyperparameters across more than 150 tasks, and was the first agent to collect diamonds in Minecraft from scratch without human data.
3.2 Value-equivalent models
MuZero [8] learns a model that predicts only what planning needs: reward, value and policy. It never reconstructs observations. Combined with tree search it matched AlphaZero in Go, chess and shogi without being given the rules, and set a new state of the art on Atari. TD-MPC [9] and TD-MPC2 [10] apply the same idea to continuous control: a latent model trained with temporal-difference targets and used for MPPI planning. TD-MPC2 trained a single 317M-parameter agent across 80 tasks.
3.3 Joint-embedding predictive architectures
LeCun [20] argued that predicting pixels wastes capacity on unpredictable detail. JEPA models predict the embedding of a target from the embedding of a context, with a slowly updated target encoder that prevents representational collapse [31]. I-JEPA [21] does this for images and V-JEPA [22] for video. V-JEPA 2 [23] pre-trains on over a million hours of video, then post-trains an action-conditioned predictor on a small amount of robot interaction data and uses it for zero-shot planning.
3.4 Generative and interactive video models
This family predicts full frames, usually with diffusion [35] or autoregressive transformers [36] over visual tokens [30]. IRIS [11] tokenises frames with a VQ autoencoder and runs a GPT-style transformer over the tokens. DIAMOND [12] showed that diffusion in pixel space preserves visual details that matter for control, and set a record for agents trained entirely inside a world model on the Atari 100k benchmark. GameNGen [13] fine-tuned Stable Diffusion to simulate the game DOOM in real time from action inputs.
Genie [14] learned a latent action space from unlabelled internet video, so that any image can become a playable 2D world. Genie 2 [15] and Genie 3 [16] extended this to 3D environments that stay consistent over long interactions. OpenAI’s Sora report [17] framed large video generators, built on diffusion transformers [37], as “world simulators”, though without explicit action inputs. UniSim [33] learned an action-conditioned simulator of real-world interaction from mixed datasets. NVIDIA Cosmos [18] packages video tokenisers, diffusion and autoregressive models as open “world foundation models” for robotics and driving, alongside driving-specific models such as GAIA-1 [19].
3.5 Trends
Three trends run across the families. Models are growing and are increasingly pre-trained on passive video before being adapted to actions. Discrete tokens and diffusion are replacing Gaussian latents as the default way to represent uncertainty. And the line between a model for an agent and a simulator for people is blurring: GameNGen and the Genie models are used both ways. A broader survey of this landscape is given by Ding et al. [40].
4. How World Models Are Trained
Training a world model means choosing four things: where the data comes from, what the model is asked to predict, which architecture carries the state, and how errors that compound over long rollouts are controlled. This section walks through each choice, then covers compute and evaluation.
4.1 Data
• Online interaction. Dreamer-style agents alternate between acting in the environment, storing transitions in a replay buffer, and training the model on sampled sequences. The ratio of gradient steps to environment steps (the replay or train ratio) is a key hyperparameter [7].
• Offline logs. Driving and robotics models are trained on large fleets of recorded trajectories with actions (steering, joint commands) [18, 19].
• Passive video. Internet video has no action labels. Genie infers a small discrete latent action between frames [14]; V-JEPA 2 learns from video first and uses a small amount of labelled robot data afterwards [23].
• Game engines and simulators. GameNGen trained on trajectories recorded by an RL agent playing DOOM [13]; simulators provide cheap, perfectly labelled data for any domain that can be scripted.
Coverage matters more than volume. A model is only trustworthy in the parts of state-action space its data visited, and a planner will find the parts it did not.
4.2 Objectives
Table 2 lists the main training objectives. Most modern systems combine several.
Table 2. Common world-model training objectives.
For the RSSM family the objective is an evidence lower bound (ELBO) over sequences. Writing h_t for the deterministic state and z_t for the stochastic latent, PlaNet and Dreamer minimise:
L = Σ_t [ −ln p(o_t | h_t, z_t) − ln p(r_t | h_t, z_t) + β · KL( q(z_t | h_t, o_t) ‖ p(z_t | h_t) ) ] (4)
DreamerV3 splits the KL term into a dynamics loss and a representation loss with different stop-gradients and weights (KL balancing), and clips each below one nat (free bits) so that the posterior is not forced onto an uninformative prior [7].
4.3 Architectures
• Recurrent state-space models (GRU + stochastic latent) remain the workhorse for control from pixels [4–7].
• Transformers over tokens or latents give longer memory and parallel training [11, 14].
• Diffusion models (U-Nets or diffusion transformers) give the sharpest frames and handle multi-modal futures, at the cost of several denoising steps per frame [12, 13, 37].
• Tokenisers (VQ-VAE [30] and causal video tokenisers [18]) compress frames or clips so the dynamics model works on short sequences.
4.4 Multi-step training and compounding error
A model trained only on one-step prediction (teacher forcing) sees perfect inputs during training but its own imperfect outputs at test time, so small errors compound. Three remedies are common. Open-loop unrolling trains on the model’s own predictions for H steps (latent overshooting in PlaNet [4]; the horizon-weighted unroll in TD-MPC [9]). Noise augmentation corrupts the context frames during training so the model learns to correct drift (GameNGen [13]). Short rollouts limit imagination to a few steps from real states when the model feeds a policy learner (MBPO [25]).
4.5 Stabilising techniques
• Symlog transforms and two-hot regression for targets with unknown scale [7].
• Free bits, KL balancing and a uniform mixture on categorical latents [6, 7].
• Stop-gradient or exponential-moving-average target encoders to prevent collapse in latent-prediction losses [9, 31].
• Normalised latents (LayerNorm, SimNorm), gradient clipping and Adam [28].
• Probabilistic ensembles to estimate epistemic uncertainty and keep planners away from unfamiliar states [24].
4.6 Using the model to act
Planning methods sample or optimise action sequences inside the model. CEM [26] fits a Gaussian to the best samples and resamples; MPPI [27] averages samples weighted by exponentiated return; MCTS [8] searches a tree over discrete actions. Imagination-based policy learning instead trains an actor and critic on λ-returns computed over imagined trajectories [5]. Dyna-style methods mix imagined transitions into the data of a model-free learner [2, 25].
4.7 Compute and scale
Scale spans six orders of magnitude. A PlaNet- or Dreamer-sized model trains on one GPU in hours to days. TD-MPC2 reaches 317M parameters [10]; Genie has 11B parameters [14]; foundation models such as Cosmos are trained on millions of hours of video on thousands of GPUs [18]. The model presented in Section 5 has about 10⁵ parameters and trains on a laptop CPU in minutes, which is the regime where the whole pipeline can be inspected and taught.
4.8 Evaluation
• Prediction quality: MSE, PSNR, SSIM and LPIPS per frame; Fréchet Video Distance for generated clips; error as a function of rollout horizon.
• Downstream control: return and sample efficiency (for example Atari 100k [12]) when the model is used for planning or policy learning.
• Planning relevance: whether the model ranks candidate actions the way the true environment would. Section 7 shows this can disagree sharply with pixel error.
• Human judgement and consistency: whether raters can tell simulated from real clips [13], and whether a revisited place looks the same [16].
5. NextWM: A Browser-Native Latent World Model
NextWM is a deliberately small world model whose whole life cycle (data collection, training, imagination and planning) runs on the web platform. Its design follows four principles:
1. Zero ML dependencies. Dense layers, backpropagation and Adam are implemented in about 230 lines of TypeScript, so every gradient can be read and checked.
2. Isomorphic code. The same modules run in a browser Web Worker and in a Next.js route handler on Node.js.
3. Planning-first objective. The model is trained by unrolling its own dynamics, so the quantity it optimises is the one the planner uses.
4. Reproducibility. All randomness comes from seeded generators; one command regenerates every number in this paper.
5.1 Environment
PointNav is a continuous-control task observed only through pixels. A point mass with momentum moves in the square [−1, 1]² under a two-dimensional force a_t ∈ [−1, 1]². Velocity decays by a factor of 0.8 per step, the force adds 0.05·a_t, and walls absorb momentum. Each episode places the agent and a goal uniformly in [−0.8, 0.8]² and lasts 50 steps. The reward is the negative Euclidean distance to the goal, and an episode counts as a success if it ends within 0.2 of the goal.
The agent sees 16×16 grayscale frames in which the agent is a Gaussian blob of intensity 1.0 and the goal a blob of intensity 0.5. Because a single frame does not show velocity, each observation stacks the current and previous frames (512 values), as in DQN [38].
5.2 Model
Figure 1. NextWM architecture. The encoder maps two stacked frames to a 16-dimensional latent. A residual dynamics network unrolls the latent for H steps under the logged actions; the decoder and reward head read every predicted latent. At decision time CEM searches over action sequences using only the dynamics and reward head.
NextWM has four multilayer perceptrons (Figure 1), 108,577 parameters in total:
• Encoder E: 512 → 128 → 16, tanh activations, giving a bounded latent z_t = E(o_t).
• Dynamics F: 18 → 128 → 16 with a residual connection, ẑ_{t+1} = ẑ_t + F([ẑ_t, a_t]). The residual form makes the identity map the default, which suits slowly changing scenes.
• Decoder D: 16 → 128 → 256, ReLU then sigmoid, reconstructing the current frame.
• Reward head R: 16 → 64 → 1.
5.3 Training objective
Each training example is a window of H + 1 observations, H actions and H + 1 rewards sampled from a replay buffer. The first observation is encoded once and the dynamics are unrolled open-loop: ẑ_0 = E(o_t) and ẑ_k = ẑ_{k−1} + F([ẑ_{k−1}, a_{t+k−1}]). The loss is
L = Σ_{k=0..H} ρ^k ℓ_k (5)
ℓ_k = λ_x ‖D(ẑ_k) − x_{t+k}‖² + λ_r (R(ẑ_k) − r_{t+k})² + 𝟙[k>0] · λ_c ‖ẑ_k − sg(E(o_{t+k}))‖² (6)
where x is the current frame, sg is the stop-gradient operator and ρ = 0.9 down-weights distant steps. The three terms play distinct roles. Reconstruction anchors the latent to the pixels and prevents the collapse to which latent-only losses are prone. Reward regression puts the task into the latent. Latent consistency, as in TD-MPC [9], keeps imagined latents on the manifold of real encodings, so that a latent reached by imagination can be decoded and scored as if it had been observed. Gradients flow through the whole unrolled chain back into the encoder.
Get Jung-Hua Liu’s stories in your inbox
Join Medium for free to get updates from this writer.
The implementation passes a finite-difference gradient check with a maximum relative error of 1.4 × 10⁻⁶ in double precision (40 random coordinates, consistency term disabled because finite differences also move its stop-gradient target). In single precision, used for speed, the check is dominated by rounding noise.
5.4 Planning in imagination
At every environment step NextWM encodes the current observation once and runs CEM [26] entirely in latent space. Each of P candidate action sequences of length 12 is rolled out through F in one batched matrix operation, scored by the sum of predicted rewards R(ẑ_k), and the 10% best candidates refit a diagonal Gaussian. After four iterations the first action of the mean is executed, and the mean is shifted by one step to warm-start the next decision. No pixels are decoded during planning. The same planner, given the true simulator instead of the model, serves as an oracle upper bound.
6. Implementation on Next.js
NextWM is a Next.js 15 application using the App Router, React 19 and TypeScript. Table 3 lists the modules; everything under lib/wm is framework-independent.
Table 3. Source layout of the NextWM repository.
6.1 Training off the main thread
The browser UI never trains on the main thread. The client component creates a module worker with new Worker(new URL(“../lib/wm/worker.ts”, import.meta.url)), which Next.js bundles as a separate chunk. The worker trains in chunks of 25 gradient steps and yields to its event loop between chunks, so stop requests are handled promptly and loss updates stream to the UI. Imagination and planning requests carry an identifier and are answered asynchronously.
6.2 Server-side training
Because the model is plain TypeScript, the route handler app/api/train/route.ts imports the same modules and trains on Node.js. The browser can load the returned weights into its worker, so a slow device can train on the server and still imagine and plan locally. Weights are exported as JSON arrays and can be downloaded from the UI.
6.3 The interactive lab
The lab page shows the true environment next to the model’s open-loop dream. After “Start dreaming”, every real action is also fed to the model, which re-imagines the whole trajectory from the snapshot observation, so the viewer watches prediction error accumulate in real time. The environment can be driven from the keyboard or by the CEM agent planning inside the model.
A production build (next build) compiles successfully; the lab page is 2.88 kB of route-specific JavaScript (105 kB first load including React and Next.js), and the model code ships in a separate worker chunk. We verified the end-to-end path in headless Chromium: training in the worker, streaming loss updates, and imagination and CEM control through the worker all ran without console errors.
7. Experiments
We ask three questions. Does NextWM learn the environment, measured as open-loop prediction error? Can a planner use it, measured as return-ranking correlation and closed-loop control? And which design choices matter?
7.1 Setup
Table 4. Experimental protocol. The ablations in Section 7.4 change only the exploration policy, λ_r and the learning rate.
Two exploration policies generate data. The OU policy applies temporally correlated noise, a_t = clip(0.7·a_{t−1} + 0.6·u_t) with u_t uniform on [−1, 1]²; its actions largely cancel, so high speeds are rare. The hold policy draws a uniform random action and holds it for 1 to 10 steps, so the data contains the sustained full-force moves a planner tends to choose.
7.2 Training dynamics
Figure 2. Training losses of the full NextWM configuration (mean over 100-step windows, log scale). Reconstruction and reward losses fall steadily. Latent consistency first rises, as the encoder spreads its latents apart, then falls. None has plateaued after 6,000 steps.
Training ran at about 12.4 gradient steps per second on one CPU thread (483 s for 6,000 steps while sharing the machine with three other runs). Reconstruction error fell from 0.0383 to 0.0047 and reward error from 0.320 to 0.017 (Figure 2).
7.3 Open-loop imagination
Figure 3. Pixel MSE of open-loop predictions against the true frame, as a function of horizon, on 40 held-out episodes (160 windows). The baseline repeats the last observed frame.
Figure 3 compares NextWM’s open-loop predictions with a copy-last-frame baseline. At short horizons the baseline wins: one step ahead its error is 0.0023 against 0.0049 for NextWM, because the decoder has a reconstruction floor while consecutive frames are nearly identical. From horizon 2 onward NextWM is better, and at horizon 15 its error is 0.0134 against 0.0229. The model has therefore learned motion, not just appearance. Figure 4 shows a held-out rollout, and it reveals a specific failure. The imagined agent blurs, as expected of a deterministic model averaging over uncertain futures, but follows the real one into the top-right corner. The goal, which never moves, fades from the imagination after about eight steps: the dynamics network does not preserve the part of the latent that encodes the static goal. Because the reward depends on the goal, this matters for planning.
Figure 4. A held-out rollout. Top: true frames. Bottom: NextWM’s open-loop imagination from the first frame under the same actions, with no further observations.
7.4 Planning, and why pixel error is not enough
Our first configuration, trained for 4,000 steps on OU data with λ_r = 1, predicted pixels reasonably but was useless for planning. Its return-ranking correlation was 0.22, and CEM control inside it reached a mean return of -42.2, no better than random actions (-41.0).
We hypothesised two causes. First, the model had rarely seen the sustained actions CEM selects, so it was queried outside its data. Second, the reward head, which is the planner’s only signal, received little weight next to 256 pixel terms. Table 5 and Figure 5 cross the two factors, with every run trained for 6,000 steps.
Table 5. Ablation over exploration data and reward weighting. Return is the undiscounted sum of rewards over 50 steps (higher is better). Control uses 30 episodes with the same start states for every row.
Figure 5. Return-ranking correlation (left) and closed-loop final distance (right) for each configuration. Dashed line: random actions; dotted line: oracle planner.
The result is clear. Pixel error at horizon 8 hardly moves across the four runs (0.0097 to 0.0106), yet ranking correlation ranges from 0.42 to 0.82. Each factor helps on its own: up-weighting the reward raises ranking to 0.56, and hold exploration raises it to 0.49. Together they are worth much more than either alone. Closed-loop control follows ranking, not pixel error: the full configuration reaches a mean final distance of 0.45 against 0.91 for random actions.
7.5 Closing the loop with planner data
Model-based agents such as PlaNet and Dreamer do not train on exploration data alone. They repeatedly act with the current model and retrain on what they see [4, 5]. We ran one such round: 100 episodes collected by CEM inside the model, with Gaussian action noise (σ = 0.3), were added to the 200 exploration episodes, and training continued for 3,000 steps. Collection took 162 s.
After this round the mean return improves from -22.4 to -20.2 and the mean final distance from 0.45 to 0.38. 4 of 30 episodes end within 0.2 of the goal, against 2 before the round; with 30 episodes this count is noisy, but return and distance move in the same direction. Interestingly, the ranking correlation on our constant-direction probe falls slightly, from 0.82 to 0.75. The planner data concentrates on short, corrective moves near the goal, so the model trades some accuracy on long straight-line moves, which the probe measures, for accuracy where decisions are made. A substantial gap to the oracle (-3.9 return, 100% success) remains; Section 8 discusses why.
7.6 Cost
A CEM decision with population 200 and horizon 12 makes 9,600 latent transitions and took about 125 ms on one CPU thread, so a 50-step episode took 6.2 s. The browser worker uses a smaller population (120, three iterations) to keep interactive control responsive. The oracle planner is faster (2 ms per decision) because the true simulator is a few arithmetic operations, whereas each model step is a pair of dense layers.
8. Discussion and Limitations
8.1 What the experiments teach
Pixel accuracy is a poor proxy for usefulness. Across our ablation, pixel error varied by a few per cent while planning quality varied several-fold. This is the argument behind value-equivalent models [8–10] and JEPA [20]: a model should be judged, and ideally trained, on the quantities its user consumes. We recommend reporting a planning-relevance metric, such as the return-ranking correlation used here, next to reconstruction metrics.
Coverage beats volume. The same 10,000 transitions helped planning much more when they contained sustained actions. Planners actively search for the model’s blind spots, so the data must cover the actions the planner will choose, or be refreshed by the planner itself (Section 7.5).
The web platform is a viable teaching and prototyping substrate. A complete world-model pipeline (data, training, imagination, planning and visualisation) fits in about 1,200 lines of TypeScript and runs in a browser tab without a GPU or a Python stack. Because the code is isomorphic, the same model can train on a Next.js server and plan on the client.
8.2 Limitations
• Toy environment. PointNav has four state variables and 16×16 frames. The findings about coverage and planning relevance are well known at scale, but the absolute numbers here say nothing about larger tasks.
• Deterministic model. NextWM has no stochastic latent and no ensemble, so it cannot represent uncertainty, and CEM can exploit its errors. An RSSM-style stochastic state [4] or an ensemble [24] is the natural next step.
• Small evaluation. Control results use 30 episodes and one training seed per configuration, without confidence intervals. Differences of a few percentage points in success rate are within noise; the ranking and distance trends are larger.
• Gap to the oracle. Even the best configuration is far from the oracle planner. Two measured causes contribute: the goal fades from long imagined rollouts (Figure 4), and the reward error is still comparable to the reward differences that decide the last few steps near the goal. Splitting the latent into a static context (the goal) and a dynamic part (the agent), or feeding the encoded goal to every dynamics step, is a direct fix.
• Speed. Plain JavaScript on one CPU thread manages about 12 gradient steps per second for this model. WebGPU kernels or TensorFlow.js [34] would raise throughput by orders of magnitude, at the cost of the zero-dependency design.
8.3 Future work
Natural extensions are a stochastic latent with KL balancing [7], an actor-critic trained in imagination to amortise planning [5], WebGPU matrix kernels, several training seeds with confidence intervals, and richer browser-rendered environments. A further direction is to use the Next.js server for heavy pre-training on passive video and the client for fast action-conditioned adaptation, mirroring the V-JEPA 2 recipe [23] at small scale.
9. Conclusion
World models now span compact latent models for control, value-equivalent planners, joint-embedding predictors and billion-parameter interactive video generators. They share one recipe: learn to predict the consequences of actions, then act by imagining them. We surveyed these families and the practical choices involved in training them. We also presented NextWM, a transparent world model that trains, imagines and plans entirely within a Next.js application. Its main lesson matches what larger systems have found: a world model is only as good as its ability to rank the choices its user will make. Reaching that ability depended on data coverage and on the loss weighting, not on pixel accuracy.
References
[1] K. Craik. The Nature of Explanation. Cambridge University Press, 1943.
[2] R. S. Sutton. Dyna, an integrated architecture for learning, planning, and reacting. ACM SIGART Bulletin, 2(4):160–163, 1991.
[3] D. Ha and J. Schmidhuber. Recurrent world models facilitate policy evolution. NeurIPS, 2018. arXiv:1803.10122.
[4] D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, J. Davidson. Learning latent dynamics for planning from pixels. ICML, 2019.
[5] D. Hafner, T. Lillicrap, J. Ba, M. Norouzi. Dream to control: Learning behaviors by latent imagination. ICLR, 2020.
[6] D. Hafner, T. Lillicrap, M. Norouzi, J. Ba. Mastering Atari with discrete world models. ICLR, 2021.
[7] D. Hafner, J. Pasukonis, J. Ba, T. Lillicrap. Mastering diverse domains through world models. arXiv:2301.04104, 2023; Nature, 2025.
[8] J. Schrittwieser et al. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588:604–609, 2020.
[9] N. Hansen, X. Wang, H. Su. Temporal difference learning for model predictive control. ICML, 2022.
[10] N. Hansen, H. Su, X. Wang. TD-MPC2: Scalable, robust world models for continuous control. ICLR, 2024.
[11] V. Micheli, E. Alonso, F. Fleuret. Transformers are sample-efficient world models. ICLR, 2023.
[12] E. Alonso, A. Jelley, V. Micheli, A. Kanervisto, A. Storkey, T. Pearce, F. Fleuret. Diffusion for world modeling: Visual details matter in Atari. NeurIPS, 2024.
[13] D. Valevski, Y. Leviathan, M. Arar, S. Fruchter. Diffusion models are real-time game engines. arXiv:2408.14837, 2024.
[14] J. Bruce et al. Genie: Generative interactive environments. ICML, 2024.
[15] Google DeepMind. Genie 2: A large-scale foundation world model. Technical blog, December 2024.
[16] Google DeepMind. Genie 3: A new frontier for world models. Technical blog, August 2025.
[17] T. Brooks et al. Video generation models as world simulators. OpenAI technical report, 2024.
[18] NVIDIA. Cosmos world foundation model platform for physical AI. arXiv:2501.03575, 2025.
[19] A. Hu et al. GAIA-1: A generative world model for autonomous driving. arXiv:2309.17080, 2023.
[20] Y. LeCun. A path towards autonomous machine intelligence. OpenReview preprint, 2022.
[21] M. Assran et al. Self-supervised learning from images with a joint-embedding predictive architecture. CVPR, 2023.
[22] A. Bardes et al. Revisiting feature prediction for learning visual representations from video (V-JEPA). arXiv:2404.08471, 2024.
[23] M. Assran et al. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv:2506.09985, 2025.
[24] K. Chua, R. Calandra, R. McAllister, S. Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. NeurIPS, 2018.
[25] M. Janner, J. Fu, M. Zhang, S. Levine. When to trust your model: Model-based policy optimization. NeurIPS, 2019.
[26] P.-T. de Boer, D. P. Kroese, S. Mannor, R. Y. Rubinstein. A tutorial on the cross-entropy method. Annals of Operations Research, 134:19–67, 2005.
[27] G. Williams et al. Information theoretic MPC for model-based reinforcement learning. ICRA, 2017.
[28] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. ICLR, 2015.
[29] D. P. Kingma and M. Welling. Auto-encoding variational Bayes. ICLR, 2014.
[30] A. van den Oord, O. Vinyals, K. Kavukcuoglu. Neural discrete representation learning. NeurIPS, 2017.
[31] J.-B. Grill et al. Bootstrap your own latent: A new approach to self-supervised learning. NeurIPS, 2020.
[32] Vercel. Next.js documentation: App Router, Route Handlers. 2025.
[33] S. Yang et al. Learning interactive real-world simulators (UniSim). ICLR, 2024.
[34] D. Smilkov et al. TensorFlow.js: Machine learning for the web and beyond. SysML, 2019.
[35] J. Ho, A. Jain, P. Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
[36] A. Vaswani et al. Attention is all you need. NeurIPS, 2017.
[37] W. Peebles and S. Xie. Scalable diffusion models with transformers. ICCV, 2023.
[38] V. Mnih et al. Human-level control through deep reinforcement learning. Nature, 518:529–533, 2015.
[39] WHATWG. HTML Living Standard: Web Workers.
[40] J. Ding et al. Understanding world or predicting future? A comprehensive survey of world models. arXiv:2411.14499, 2024.
