Since Hinton has actually replied let me clarify some things - LLMS don’t *always* regurgitate - LLMs don’t literally store full texts - but…

Since Hinton has actually replied let me clarify some things - LLMS don’t always regurgitate - LLMs don’t literally store full texts - but given the mechanisms that they use they do sometimes regurgitate long passes through their process of resynthesis - the evidence for that is overwhelming - in the literature it is called “memorization”; that’s a metaphor, but it effectively works out to be something much like that.

Source: Gary Marcus (X) | 2026-05-11