Pretraining large language models with MXFP4
arXiv:2605.09825v1 Announce Type: cross Abstract: Why does full-pipeline FP4 training of large language models often diverge, even when forward activations and activation gradients remain stable? We a
Knowledge catalogue
arXiv:2605.09825v1 Announce Type: cross Abstract: Why does full-pipeline FP4 training of large language models often diverge, even when forward activations and activation gradients remain stable? We a
arXiv:2510.20129v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) remain vulnerable to jailbreak attacks, where adversarially crafted prompts induce policy-violating responses des
arXiv:2605.09005v1 Announce Type: cross Abstract: Vision-Language-Action models (VLAs) support generalist robotic control by enabling end-to-end decision policies directly from multi-modal inputs. As
arXiv:2605.08974v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have advanced video understanding, they remain highly prone to hallucinations in dynamic scenes. We arg
arXiv:2605.08168v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models offer a promising path to generalist robot control, but their inference latency causes observation staleness when
arXiv:2605.10158v1 Announce Type: new Abstract: Process Reward Models (PRMs) are a powerful mechanism for steering large language model reasoning by providing fine-grained, step-level supervision. How
arXiv:2605.07488v1 Announce Type: new Abstract: The scaling of Large Multimodal Models (LMMs) is constrained by the quality-quantity trade-off inherent in synthetic data. Previous approaches, such as
arXiv:2605.08063v1 Announce Type: cross Abstract: Existing Flow Matching (FM) text-to-image models suffer from two critical bottlenecks under multi-task alignment: the reward sparsity induced by scala
Introducing Daybreak: frontier AI for cyber defenders. Daybreak brings together the most capable OpenAI models, Codex, and our security partners to accelerate cyber defense and continuously secure sof
It is one reason why I think the push for smaller, local models is more complicated than people think. If you want good answers, especially good answers to unexpected problems, frontier models will ge
arXiv:2605.07794v1 Announce Type: new Abstract: World Action Models (WAMs) are an emerging family of policies that tie robot action generation to future-observation modeling. In this work, we focus on
arXiv:2605.08078v1 Announce Type: new Abstract: Diffusion-based models decompose sampling into many small Gaussian denoising steps -- an assumption that breaks down when generation is compressed to a
arXiv:2605.07496v1 Announce Type: new Abstract: Bird's-eye-view (BEV) images have been widely demonstrated to provide valuable prior information for navigation. Given the global information provided b
arXiv:2605.07574v1 Announce Type: new Abstract: Mainstream vision-language models (VLMs) fundamentally struggle with severe optical ambiguities, such as reflections and transparent objects, due to the
arXiv:2605.07554v1 Announce Type: cross Abstract: Protein language models are trained primarily with masked language modeling (MLM), which predicts amino-acid identities at masked positions. We ask wh
arXiv:2605.06901v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly shaping the private and professional lives of users, with numerous applications in business, education, fi
arXiv:2602.17251v2 Announce Type: replace Abstract: Electroencephalography (EEG) foundation models (EFMs) have shown strong potential for transferable representation learning, yet their adaptation in
arXiv:2510.01290v2 Announce Type: replace Abstract: The long-output context generation of large reasoning models enables extended chain of thought (CoT) but also drives rapid growth of the key-value (
The great thing is that the names are so baffling that the most important models OpenAI released were names davinci-002, GPT-3.5, GPT-4, o1-preview, o3, GPT-5 Pro, and you would never know the ways th
Training models involves many technical and social processes, so prevention of CoT grading has to be built into the process. We’re improving real-time CoT-grading detection, safeguards against acciden
arXiv:2605.05090v1 Announce Type: new Abstract: We present an automated, contrastive evaluation pipeline for auditing the behavioral impact of interventions on large language models. Given a base mode
arXiv:2512.06393v5 Announce Type: replace-cross Abstract: Large language models (LLMs) achieve high accuracy on many reasoning benchmarks but remain brittle under structural perturbations of rule-base
arXiv:2605.04413v1 Announce Type: new Abstract: Structural causal models provide a unified semantics for interventions and counterfactuals, but most identifiability results rely on restrictive assumpt
I don't really ever trust benchmarks, so ocassionally I'm stress-vibe-testing a bunch of new models on some very complex agent work (hundreds of tools, not your simple coding agent stuff) and to my su
arXiv:2510.04166v2 Announce Type: replace-cross Abstract: Syntax highlighting is a critical feature in modern software development environments, enhancing code readability and developer productivity.
New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read
Open Models Make Agentic Batch Processing Economically Viable A lot of world’s work looks like “Do X for EVERY Y” - read every trace - respond to every email - deep dive into every document - enrich e
arXiv:2605.05159v1 Announce Type: new Abstract: We present our system for SemEval-2026 Task 9: Multilingual Polarization Detection, a binary classification task spanning 22 languages. Our approach fin
arXiv:2605.04539v1 Announce Type: new Abstract: Direct Preference Optimization (DPO), the efficient alternative to PPO-based RLHF, falls short on knowledge-intensive generation: standard preference si
arXiv:2602.02315v2 Announce Type: replace Abstract: Large language models (LLMs) form implicit beliefs (posteriors over latent variables) from prompts, but we lack a mechanistic account of how these b
arXiv:2605.04874v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has proven to be an effective solution for mitigating hallucination in Multimodal Large Language Models (MLLMs) b
We recently found some instances of CoT grading during the training of previously deployed models after building a system that scans all OpenAI RL runs for accidental CoT grading. We did not find clea
arXiv:2605.03403v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has recently shown strong performance in post-training large language models and vision-language models. It ra
arXiv:2510.09472v2 Announce Type: replace Abstract: Despite the remarkable progress in neural models, their ability to generalize, a cornerstone for applications such as logical reasoning, remains a c
arXiv:2505.16527v2 Announce Type: replace Abstract: Building generative models for relational databases (RDBs) is important for many applications, such as privacy-preserving data release and augmentin
arXiv:2605.01790v1 Announce Type: cross Abstract: A common design pattern in high-quality music generation is to handle structure and fidelity in different representation spaces: a generator first mod
arXiv:2605.01694v1 Announce Type: new Abstract: A world model matters to an agent only through the state it constructs. That state must preserve some information, discard other information, and suppor
arXiv:2605.03485v1 Announce Type: new Abstract: Multidimensional human understanding is essential for real-world applications such as film analysis and virtual digital humans, yet current LVLM benchma
MRC is already deployed across all of OpenAI’s largest supercomputers that we use to train frontier models, including our site with @Oracle Cloud Infrastructure (OCI) in Abilene, Texas, and in @Micros
arXiv:2605.03301v1 Announce Type: new Abstract: De-identification of clinical text remains essential for secondary use of electronic health records (EHRs), yet public benchmarks such as i2b2 2006/2014
arXiv:2603.04601v2 Announce Type: replace-cross Abstract: Code generation has emerged as one of AI's highest-impact use cases, yet existing benchmarks measure isolated tasks rather than the complete '
arXiv:2605.02263v1 Announce Type: new Abstract: Recent diffusion large language models (dLLMs) have demonstrated both effectiveness and efficiency in reasoning via a block-based semi-autoregressive ge
arXiv:2605.00731v1 Announce Type: cross Abstract: While Graph Foundation Models (GFMs) have achieved remarkable success in homogeneous graphs, extending them to multi-domain heterogeneous graphs (MDHG
arXiv:2605.00371v1 Announce Type: cross Abstract: In this paper, we propose GaMMA, a state-of-the-art (SoTA) large multimodal model (LMM) designed to achieve comprehensive musical content understandin
Google DeepMind, Microsoft, and Elon Musk's xAI have agreed to allow the US government to review new AI models before they're released to the public. In an announcement on Tuesday, the Commerce Depart
arXiv:2605.01627v1 Announce Type: new Abstract: Low-rank decomposition is a compelling approach for compressing large language models, but its effectiveness hinges on selecting which singular-vector b
arXiv:2605.01822v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being used to support scientific discovery. In chemistry, tasks such as reaction prediction and structure
arXiv:2605.00877v1 Announce Type: cross Abstract: The vast and underexplored ocean plays a critical role in regulating global climate and supporting marine biodiversity, yet artificial intelligence ha
arXiv:2605.02623v1 Announce Type: new Abstract: Video Moment Retrieval (VMR) aims to localize temporal segments in videos that correspond to a natural language query, but typically assumes only a sing
arXiv:2605.01817v1 Announce Type: new Abstract: Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the
arXiv:2603.22097v2 Announce Type: replace-cross Abstract: Foundation models are now increasingly being developed for Earth observation (EO), yet they often rely on stochastic masking that do not expli
arXiv:2605.01911v1 Announce Type: new Abstract: Purpose: Vision-language models (VLMs) have shown promising performance in surgical visual question answering (VQA). However, existing surgical VQA data
arXiv:2504.20605v2 Announce Type: replace Abstract: Moral stories are a time-tested vehicle for transmitting values, yet modern NLP lacks a large, structured corpus that couples coherent narratives wi
arXiv:2605.00902v1 Announce Type: new Abstract: Foundation models are reshaping computational histopathology, yet their value for whole-slide image retrieval relative to strong patch-based and supervi
we have very efficient models, especially for their capability level happy codexing yo, i'm actually worried. codex limits are genuinely insane so it's sus af .. i feel this is an intentional move for
A Reddit discussion from the Ollama community addresses reports of slow model responses, clarifying that the models themselves are not inherently slower but that rate limiting is a more likely cause o
arXiv:2411.10915v2 Announce Type: replace Abstract: Large Language Models (LLMs) have revolutionized natural language processing, but their susceptibility to biases poses significant challenges. This
arXiv:2602.00665v3 Announce Type: replace Abstract: Customer-service question answering (QA) systems increasingly rely on conversational language understanding. While Large Language Models (LLMs) achi
arXiv:2406.14429v3 Announce Type: replace-cross Abstract: In the landscape of generative artificial intelligence, diffusion-based models have emerged as a promising method for generating synthetic ima
arXiv:2510.18900v2 Announce Type: replace-cross Abstract: Accurate prediction of atomistic, thermodynamic, and kinetic properties from molecular structures underpins materials innovation. Existing com