Understanding the Parameter Space Geometry of Transformers Encoding Boolean Functions
DGX agentarXiv:2606.08768v1 Announce Type: new Abstract: Transformers consistently fail to learn certain simple functions that are provably expressible with specific parameter settings. This gap between learna