Cool idea from Nous Research. What if you could speed up long-context pretraining with a subquadratic wrapper that you remove before deploym…
DGX agentCool idea from Nous Research. What if you could speed up long-context pretraining with a subquadratic wrapper that you remove before deployment? That is the idea behind Lighthouse Attention. The metho