World Models Explained: How Machines Predict Reality

Ask most people what an AI system knows, and they picture a chatbot predicting the next word. But a growing branch of research argues that real intelligence needs something else entirely: an internal picture of how the world works, one that can predict what happens next when something acts on it. That idea sits at the heart of AI world models, a paradigm that moved from a niche reinforcement learning technique to one of the most watched frontiers in the field. This explainer walks through what these systems are, where the idea came from, the two rival approaches now competing, and why some of the most prominent researchers believe world models, not larger language models, are the road to machines that genuinely understand.

What AI world models are

According to Wikipedia's overview, a world model is a machine learning system that builds an internal representation of an environment and predicts how that environment changes over time in response to actions. That last clause is the whole point. A world model is not just describing what it sees; it is running a mental simulation of what would follow if an agent did something. Push a virtual car left, and a good world model can anticipate where it ends up, without touching the real road.

This makes world models fundamentally different from large language models. An LLM predicts text tokens and, as critics point out, lacks any built-in understanding of physics. A world model instead operates on sensor data such as video or lidar, and it is designed to support planning and causal reasoning about a physical or simulated space. The distinction is not merely academic. It is the reason a language model can write a fluent paragraph about gravity while failing to predict whether a stacked tower of blocks will topple, a task a competent world model is built to handle.

Where the idea of world models came from

The concept is older than the current excitement. Jürgen Schmidhuber introduced an early version in 1990, proposing recurrent neural networks that predict future states from observations in order to train agents. The idea then waited nearly three decades for the compute and data to catch up. In 2018, David Ha and Schmidhuber published the paper that gave the paradigm its modern name and momentum, showing agents that learned to drive virtual cars and play video games inside self-generated simulations.

Their design broke the system into clear parts, often summarized as vision, memory, and control. A vision component compresses raw observations into a concise representation. A memory component predicts how that representation will evolve over time. A controller then uses those predictions to choose actions. The striking result was that an agent could train largely inside its own imagined version of the environment and still perform well when returned to the real task, reaching high scores in a car-racing simulation and a navigable 3D game world. That demonstration turned world models into a prominent thread in the model-based reinforcement learning community.

The reinforcement learning framing matters for understanding why researchers care. Model-free methods learn purely by trial and error, repeating an action many times to discover its consequences. Model-based methods, powered by a world model, let an agent simulate consequences internally before acting. That internal rehearsal is far cheaper and safer than learning everything through real attempts, which is exactly why the approach became attractive for robotics and any domain where mistakes are costly.

The two competing paths for world models

Today the field splits into two broad philosophies, and the split is the most important thing to understand about the current moment. One camp predicts the future in an abstract latent space, without drawing every pixel. The other generates the future frame by frame as visible video.

The latent, predictive approach is championed by Yann LeCun, whose 2022 position paper proposed the Joint Embedding Predictive Architecture, or JEPA. The insight is that predicting raw pixels is wasteful and often impossible, because the future contains too many irrelevant details a model can never get exactly right. Instead, a JEPA-style model predicts a compressed representation of what comes next, focusing on the structure that matters for reasoning while ignoring pixel-level noise. Wikipedia's summary lists the typical components of such systems: an encoder that turns observations into embeddings, a predictor that estimates future embeddings, an optional critic that selects among predictions, and a regularizer that keeps the embeddings stable.

The generative approach goes the opposite way. Systems such as DeepMind's Genie and OpenAI's Sora accept a prompt or a layout and output consistent video or 3D scenes, rendering the imagined world in full visual detail. Both arrived in public view in February 2024, and together they pulled world models out of the lab and into mainstream attention. The tradeoff between the two paths is real. Generative models produce vivid, inspectable output that humans can watch and judge, but at high computational cost. Latent models are leaner and better suited to fast planning, but their internal predictions are not something a person can simply look at.

DeepMind's Genie series in detail

Google DeepMind's Genie line is the clearest public example of the generative branch maturing quickly. According to the Genie overview, the original Genie launched in March 2024 as a spatiotemporal transformer with 11 billion parameters, trained on video game footage and capable of generating two-dimensional environments at roughly one frame per second. It learned interactive worlds from unlabeled video, meaning it picked up how environments respond to actions without being explicitly told.

Genie 2 followed in December 2024, extending the system to three-dimensional environments with isometric and first-person perspectives. It generated worlds lasting 10 to 20 seconds at 360p, a modest window but a clear step up in spatial coherence. Then came Genie 3 in August 2025, described as capable of generating and modifying interactive worlds in real time. Its specifications mark the jump plainly: 720p resolution at 24 frames per second, with a memory that holds a coherent world for about one minute rather than a handful of seconds. DeepMind also connected it to Google Street View for generating street scenes, and opened access through a Project Genie web interface for AI Ultra subscribers starting January 29, 2026.

LeCun's bet against language models

The world models debate is also a debate about the limits of the technology that currently dominates AI. Yann LeCun has become the most visible advocate for the view that large language models trained only on text will never reach general intelligence, and that the path forward runs through world models instead. His argument is direct: a model that only predicts text cannot predict real-world events, because it has no grounding in how physical systems actually behave.

Meta's work under his direction gave that thesis a concrete artifact. V-JEPA 2, released in 2025, reached leading performance on video understanding and physical reasoning at the time and supported zero-shot robot control in unfamiliar environments, meaning it could guide a robot in settings it had not been trained on. Reporting on the field describes LeCun's AMI Labs as the largest bet yet on the idea that understanding, not fluency, is the real target, and that world models are how you get there. Whether he is right remains an open scientific question, but the position has pulled serious talent and funding toward the predictive approach.

Large language models trained only on text cannot predict real-world events, because they lack any grounding in physics.The world models thesis, as argued by Yann LeCun

What world models are used for

The practical appeal becomes obvious once you see where these systems land. In robotics, training inside a learned simulation lets a machine attempt thousands of actions without risking damage in the real world, then transfer what it learned to physical hardware. In autonomous driving, Waymo's world model, noted in coverage of the field in 2026, can simulate rare and dangerous situations such as unusual pedestrian behavior, generating the edge cases that are too scarce or too risky to collect on real roads.

Nvidia has pushed the idea toward physical AI with Cosmos 3, a 2026 system that combines physical reasoning with world simulation and action generation, aimed at giving robots and vehicles a shared predictive substrate. On the consumer side, Genie 3 lets people generate playable, interactive environments from a short text description, hinting at a future for games and creative tools where the world itself is generated on demand. Scientists are also exploring world models to simulate physical and biological systems at scale, and logistics teams use accurate digital twins to test strategies before committing to them. The common thread is prediction as a service: any task that benefits from rehearsing consequences before acting is a candidate.

The limits that benchmarks keep exposing

For all the momentum, honest evaluation reveals that world models are still far from a solved problem, especially on the physical understanding they are supposed to deliver. The most sobering result comes from physics benchmarks. On IntPhys 2, a test built to check whether a model can detect violations of physical law, Meta's V-JEPA 2 performed little better than random chance on many conditions. A system marketed for physical reasoning that cannot reliably tell when physics has been broken is a clear signal of how early this all remains.

There is also a measurement problem lurking underneath the leaderboards. Pixel reconstruction error, a common way to score generative world models, often fails to predict real-world performance. A model can render a plausible-looking future that is subtly wrong in ways a pixel-difference metric never catches, which means the field partly lacks reliable yardsticks for the very capability it is chasing. Researchers have started to respond, and recent work such as the 2026 paper "A Definition and Roadmap for World Models" (available on arXiv) tries to pin down definitions and evaluation more rigorously, a sign the community knows the hype has outrun the measurement.

Video generators as accidental world simulators

One of the more provocative claims in the field is that a good enough video generator has to learn a world model whether or not anyone designed it to. The reasoning is intuitive. To predict the next frame of a video convincingly, a system has to internalize object permanence and rough physics, plus how scenes respond to motion, because a model that ignored those regularities would produce output that looks broken. OpenAI leaned into this framing when it introduced Sora in February 2024, presenting large video generation models as a path toward general-purpose simulators of the physical world.

That claim is contested, and the disagreement is instructive. Skeptics note that producing a plausible-looking clip is not the same as holding a consistent, queryable model of physics, and that video generators frequently violate physical law in subtle ways when pushed. The gap between generating something that looks right and understanding why it is right is exactly where the two camps in the field part company. Generative systems like Sora and Genie bet that scale and visual prediction will squeeze out a usable world model as a byproduct. Predictive systems like JEPA bet that you have to target the underlying structure directly, in a latent space, rather than hoping it emerges from pixel prediction. Which bet pays off is still genuinely unresolved, and it is one of the most interesting open questions in the discipline.

Why world models are having a moment now

The idea is decades old, so it is worth asking why it surged into prominence in late 2025 and early 2026 rather than earlier. Part of the answer is raw capability. The compute and the architectures that made large-scale video and representation learning practical simply did not exist when Schmidhuber first sketched the concept in 1990. Self-supervised learning on enormous amounts of unlabeled video changed what a world model could be trained on, since the internet supplies effectively unlimited footage of the physical world behaving as it does. That shift is easy to underrate. For most of the field's history the bottleneck was not the idea but the fuel, and cheap, abundant video removed it almost overnight.

Money followed the same logic. Once the technical pieces were in place, capital moved toward teams promising machines that understand physical reality rather than only text, and dedicated world model startups began raising large rounds through 2026. The result is a rare alignment where a long-standing research idea, the data to feed it, and investor appetite arrived at the same time, which tends to compress years of expected progress into a much shorter window.

The other part is a sense that text-only systems may be approaching a ceiling for certain kinds of reasoning. As language models grew astonishingly fluent without gaining reliable physical or spatial understanding, the argument that fluency and understanding are different things gained force. World models became the obvious candidate for supplying what text pretraining does not, which is grounding in how an environment actually responds to action. That is why so much attention, from DeepMind's Genie work to Meta's JEPA program to Nvidia's Cosmos 3 in 2026, converged on the same paradigm at roughly the same time.

How to think about world models going forward

World models are best understood not as a finished product but as a wager about what intelligence requires. The wager is that fluency with language, however impressive, is not the same as understanding a world you can act in, and that closing the gap means building systems whose core job is prediction of consequences rather than prediction of text. The evidence so far is genuinely mixed. Genie 3's real-time interactive worlds and V-JEPA 2's zero-shot robot control show the approach producing capabilities that pure language models do not, while the IntPhys 2 results show how shallow the physical grounding can still be.

For anyone tracking where AI is headed, the useful move is to watch the boring metrics rather than the flashy demos. Memory duration and physical-reasoning benchmark scores will say more about progress than any single impressive video, as will successful transfer from simulation to real hardware. If the predictive camp is right, the models that matter most in a few years may not be the ones that talk best, but the ones that most accurately imagine what happens next. And if the benchmarks keep exposing gaps as sharply as IntPhys 2 did, the honest conclusion is that world models are a promising direction still in search of the understanding they promise.