Research
A frozen transformer learned that wombats produce cube shaped droppings and still knows after cold reload [R]
This Reddit post from r/MachineLearning discusses an experiment in which a transformer model was fine-tuned or prompted to encode the niche biological fact that wombats produce cube-shaped droppings —
This Reddit post from r/MachineLearning discusses an experiment in which a transformer model was fine-tuned or prompted to encode the niche biological fact that wombats produce cube-shaped droppings — a trait unique among mammals, caused by non-uniform elasticity in the wombat's intestines. The post explores whether this specific learned knowledge is retained after the model's weights are frozen and reloaded from disk ("cold reload"), serving as a case study in knowledge persistence and the reliability of frozen model states. It likely sparked community discussion about how factual knowledge is stored and recovered in transformer architectures across serialization/deserialization cycles.
Related
- PhD or Masters for Computational Cognitive Science [R]
- FlashAttention (FA1–FA4) in PyTorch - educational implementations focused on algorithmic differences [P]
- Educational PyTorch repo for distributed training from scratch: DP, FSDP, TP, FSDP+TP, and PP [P]
Source: r/MachineLearning | 2026-04-12