The short version
- Separate JEPAs can learn the same representation up to a rotation when each is trained on good views of the same scene, with no image-text pairs.
- Next-token prediction is treated as a bad-view JEPA, which is why language models mostly buy visual alignment by getting larger.
- A 109M-parameter text JEPA beat language models up to 72B on cross-modal alignment and matched that alignment with 42 times fewer training samples.
At a glance
| Authors | Yann LeCun and Randall Balestriero |
|---|---|
| Where to read it | Paper page on alphaxiv.org |
| Text JEPA size | 109 million parameters |
| Sample efficiency | 42 times fewer samples than next-token prediction |
| Category | AI & Models |
| Creator / publisher | alphaXiv |
| Official link | alphaxiv.org ↗ |
| Heat | 🔥 177.9K Heat |
| Announced | |
| Data updated |
When separate models learn one world
We turn the hypothesis into a theorem.
The Platonic Representation Hypothesis says models trained on different modalities, such as images and text, converge on a similar representation of the world. LeCun and Balestriero give that idea a guarantee instead of leaving it as a scaling conjecture. Under their conditions, two JEPAs trained completely separately learn essentially the same underlying representation.
One model sees different images of the same scene. The other sees different descriptions of that scene. They are never trained on image-text pairs. The key requirement is a good view: positives are two observations of the same scene, and any remaining mismatch stays bounded. When that holds, the learned representations match except for a rotation. The hypothesis becomes both a proof and a design rule.
Why next-token prediction is a bad view
The paper treats next-token prediction as a JEPA with a poor choice of positives: fragments of the same document. That framing explains why a language model’s alignment with vision tends to arrive through scale. Making the model larger is a workaround for a weak training signal, not proof that scale is the only path.
They check the theory from 145,000 captions up to the texts paired in the study with 940 million web images. The resulting scaling law of alignment predicts unseen runs within seed noise. A good view is the lever that improves alignment without relying on size alone.
- Good view
Two observations of the same scene, with other discrepancies bounded, so separate JEPAs align up to a rotation.
- Bad view
Next-token prediction, where positives are pieces of one document and visual alignment is left to scale.
- What the law adds
Alignment scaling fitted on these runs predicts held-out runs inside ordinary seed noise.
A small text model that lines up better
On the same corpus, good views reach the same image-text alignment with 42 times fewer training samples than next-token prediction. A 109-million-parameter text JEPA aligned more closely with DINOv2 and CLIP image encoders than the language models they evaluated, up to 72 billion parameters, and more closely than CLIP’s text encoder.
A single rotation between the separately trained encoders is enough to build an image captioner, with CIDEr .838 on COCO. That rotation does not need paired data. On the latest pair-free benchmark, the text JEPA beat every zero-shot accuracy reported for language models up to 8 billion parameters: 64 percent top-5 on ImageNet-100, against at most 56 percent.
| Comparison | Result in the paper |
|---|---|
| Training samples for equal alignment | Good views use 42 times fewer than next-token prediction |
| Cross-modal alignment | 109M text JEPA vs language models up to 72B, plus CLIP’s text encoder |
| COCO captioning via one rotation | CIDEr .838 |
| ImageNet-100 top-5, pair-free | 64 percent vs at most 56 percent for language models up to 8B |
What is still conditional
The theorem is not a claim that every training run collapses to one embedding. It holds when the view is good and leftover discrepancies are bounded, and the shared representation is identified only up to a rotation. The alphaxiv.org page is the official write-up of the paper. Nothing in the announcement specifies public weights, an API, or a price.
The practical bet is view design. If your positives are not two observations of the same scene, the shortcut does not apply, and the paper’s own account says scale becomes the expensive substitute. How far the same view rule travels beyond the image and text setups they validate is left open by the results reported here.
FAQ
Do the image and text JEPAs need paired training data?
No. Each model trains only on its own modality. One rotation estimated between the finished encoders is enough for captioning, and the paper says that rotation needs no pairs.
Does the theorem say any two models will match if you scale them?
No. Matching, up to a rotation, is guaranteed under the good-view conditions. Next-token prediction is classified as a bad view, so those language models are not given that guarantee.
