Research
Do you want to know why LLMs feel sharp on surface semantics but hollow on the fine-grained stuff? “From Tokens to Thoughts: How LLMs and Hu…
Do you want to know why LLMs feel sharp on surface semantics but hollow on the fine-grained stuff? “From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning” Come by the poster tomor
Do you want to know why LLMs feel sharp on surface semantics but hollow on the fine-grained stuff? “From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning” Come by the poster tomorrow at #ICLR2026 - Fri, Apr 24, 6:30–9:00 AM PDT (10:30AM local time!), Pavilion 3, P3-#1017 to know the answer! We think it's because they overcompress. Humans keep "inefficient" concepts due to nuance. LLMs discard those for cleaner, information-theoretic compression. Different objectives yield different representations. Using an Information Bottleneck lens across 40+ models, we also found that encoders align with humans better than decoders many times their size, and that during training, semantic processing migrates from deep layers to mid-network as the model discovers sparser encodings. @ChenShani2 Liron Soffer @jurafsky @ylecun
Related
- Yann LeCun (@ylecun ): Sillion Valley is 'completely LLM-pilled' 'In the end, if you’re interested in building systems that have the intelli…
- This 2 hour lecture by Yann LeCun (Turing Award winner) will teach you why the next trillion dollar AI company won't be built on LLMs. He tr…
- “AI skeptic” v “AI cynic” is going to be an important distinction in the next few years, IMO. I am not a skeptic about AI abilities but I am…
- 全网都在吹的LeCun新论文，90%的解读都是错的。 他们说生成式AI是死路，说过去三年花的几百亿全白费了，说15M参数的小模型就能吊打万亿大模型。 这些全是营销号的夸张， 我觉得这篇论文的真正分量比他们吹的还要重。 Yann LeCun团队这次解决了JEPA困扰了好几年的表征坍…
Source: Yann LeCun (X) | 2026-04-23