OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling
arXiv:2608.05141v1 Announce Type: new Abstract: Context lengths of language models (LMs) have dramatically increased, driven by the demands for in-context learning, self-improvement, and long-horizon