Search: “open-weights”

Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models

arXiv:2606.26079v1 Announce Type: new Abstract: Standard benchmarks for multimodal large language models (MLLMs) score each item on one canonical ordering and miss whether order-irrelevant shuffling c

25 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

DGX agent

arXiv:2606.24083v1 Announce Type: cross Abstract: 'Talk short. Drop grammar. Save token.' This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything d

model-releasesarxiv-cs-ai

24 Jun 2026

EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding

arXiv:2606.24422v1 Announce Type: new Abstract: We introduce EgoSAT, the first comprehensive benchmark for egocentric video reasoning in streaming settings, designed to evaluate the capabilities of mo

GeoT2V-Bench: Benchmarking 3D Consistency in Text-to-Video Models via 3D Reconstruction

arXiv:2606.24829v1 Announce Type: new Abstract: Camera-prompted text-to-video (T2V) models are increasingly used to synthesize virtual camera captures, such as orbiting objects or moving through stati

Local Ai

Ill-Posed by Design: Probing Evidence Use in VLMs

arXiv:2606.24335v1 Announce Type: new Abstract: Counterfactual analysis is widely used to study evidence use in vision-language models, but its diagnostic value is limited on well-posed tasks: when se

local-aiarxiv-cs-cv

On the Stability of Prompt Ranking in Large Language Model Evaluation

arXiv:2606.24381v1 Announce Type: cross Abstract: Prompt-based interaction has become a dominant paradigm for using large language models (LLMs), where multiple candidate prompts are evaluated and the

Agents

Unlimited OCR is a great model on table parsing and understanding proper reading order. However it does struggle a little on semantic format…

Unlimited OCR is a great model on table parsing and understanding proper reading order. However it does struggle a little on semantic formatting, charts (it does decent at bounding boxes). Attaching t

agentsjerry-liu--x

We've provided some updated results on Mistral OCR that make use of the annotation feature for charts. The overall score is ahead of GPT-5.5…

model-releasesjerry-liu--x

We've provided some updated results on Mistral OCR that make use of the annotation feature for charts. The overall score is ahead of GPT-5.5 and just behind Gemini 3.1 Pro, which is quite impressive f

Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models

arXiv:2606.21645v1 Announce Type: cross Abstract: Large language models (LLMs) can readily reproduce conventional expressions, yet their ability to model gradient frequency distributions remains under

safetyarxiv-cs-lg

CFAgentBench: A Reproducible Environment and Benchmark for Autonomous Construction-Finance Agents

model-releasesarxiv-cs-lg

arXiv:2606.22000v1 Announce Type: cross Abstract: We introduce CFAgentBench, a reproducible, self-hostable environment and benchmark for autonomous construction-finance agents: a CFO/controller-class

Coherence Under Commitment: Probing Generalization and Vacuous Memorization in LLM Logical Reasoning

arXiv:2606.21083v1 Announce Type: cross Abstract: Large language models (LLMs) deployed for logical reasoning in knowledge-intensive domains exhibit a subtle but critical failure: coherence can be vac

researcharxiv-cs-lg

Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness

arXiv:2606.15980v2 Announce Type: replace Abstract: Activation monitors -- lightweight probes trained on a language model's internal representations -- are an increasingly common layer in deployment s

safetyarxiv-cs-lg

Does RoPE Prevent or Degrade Retrieval Heads? A Mechanistic Analysis Across Model Families

model-releasesarxiv-cs-lg

arXiv:2606.21249v1 Announce Type: new Abstract: Retrieval heads, attention heads that copy information from earlier context to the current position, have been proposed as the mechanistic substrate for

GeoFidelity-Bench: Evaluating Segment-Level Geographic Fidelity in Text-to-Image Street-View Generation

arXiv:2606.23669v1 Announce Type: new Abstract: Text-to-image models can generate visually plausible city streets, but whether their outputs correspond to a requested road segment rather than a generi

Hard or Just Unreached? Diagnosing the Sampling Blind Spot in Math-Reasoning Difficulty Estimation

arXiv:2606.19636v2 Announce Type: replace Abstract: Math and science reasoning benchmarks rely on pass@k, the fraction of sampled chains that reach gold, as the canonical per-example difficulty signal

researcharxiv-cs-lg

Tutorials

GLM-5.2 is great at design (Opus level IMO). I am also starting to see great results with long-running tasks, too. How is this possible? I t…

GLM-5.2 is great at design (Opus level IMO). I am also starting to see great results with long-running tasks, too. How is this possible? I think there are a few clever hacks. But I just came across th

tutorialsdair-ai--x

20 Jun 2026

I have some very big news... KernelBench-Hard with H100 and B200 (single gpu results) AND KernelBench-Mega tested on RTX PRO 6000, H100, B20…

model-releasesclem-delangue--x

I have some very big news... KernelBench-Hard with H100 and B200 (single gpu results) AND KernelBench-Mega tested on RTX PRO 6000, H100, B200 is finally out! Starting with Mega, each of models wrote a

20 Jun 2026

BioMamba: Domain-Adaptive Biomedical Language Models

arXiv:2408.02600v3 Announce Type: replace Abstract: Background. Biomedical language models should improve performance on biomedical text while retaining general-language-modeling fluency. For Mamba-ba

11 Jun 2026

Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data

arXiv:2606.11961v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as conditional generators for structured data, relying on in-context learning (ICL) to adapt to new

11 Jun 2026

i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models

arXiv:2606.11289v1 Announce Type: new Abstract: Diffusion models have consistently driven progress in text-to-image generation. However, it is challenging to attribute recent progress to specific mode

11 Jun 2026

As vertically integrated platforms start to dominate they lock out third party access to the most valuable portions of the platform. Of cour…

As vertically integrated platforms start to dominate they lock out third party access to the most valuable portions of the platform. Of course, Anthropic is has the right to implement whatever policy

safetytogether-ai--x

DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation

arXiv:2606.10142v1 Announce Type: new Abstract: Recent advances in 3D generation have led to substantial improvements in realism, controllability, and efficiency, yet the evaluation of 3D assets remai

GhazalBench: Evaluating LLM Understanding and Canonical Surface-Form Access in Persian Ghazals

arXiv:2603.09979v2 Announce Type: replace Abstract: Persian poetry plays an active role in Iranian cultural practice, where verses by canonical poets such as Hafez are frequently quoted, paraphrased,

Mechanistic Analysis of Alignment Algorithms in Language Models

arXiv:2606.09850v1 Announce Type: cross Abstract: Post-training alignment algorithms are predominantly evaluated as black boxes, obscuring how they reshape language models' internal computations. We p

safetyarxiv-cs-cl

T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains

arXiv:2606.11070v1 Announce Type: cross Abstract: Recent advances in reasoning and tool-calling capabilities of large language models (LLMs) have enabled increasingly capable agentic systems. However,

VFUSE: Virulent Feature Understanding with Sparse autoEncoders

arXiv:2606.10080v1 Announce Type: cross Abstract: Generative models have shown remarkable progress in a variety of domains such as protein design, but such power enables the opaque generation of hazar

researcharxiv-cs-ai

WebChallenger: A Reliable and Efficient Generalist Web Agent

arXiv:2606.10423v1 Announce Type: new Abstract: Autonomous web navigation remains challenging for LLM agents, and the strongest generalist systems rely on proprietary reasoning models whose inference

GRPO Does Not Close the Multi-Agent Coordination Gap

model-releasesarxiv-cs-lg

arXiv:2606.07845v1 Announce Type: cross Abstract: We measure how well current large language models coordinate as multiple agents sharing a common resource, using the dining philosophers problem as a

OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics

arXiv:2606.09826v1 Announce Type: cross Abstract: Vision-language model (VLM) agents are increasingly deployed in interactive game environments. Yet game benchmarks for VLM agents typically report a s

Payoff scaling shapes cooperation in LLM agents across languages

arXiv:2601.19082v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents that negotiate, coordinate, and act on behalf of users. Whether they coo

safetyarxiv-cs-ai

Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override

arXiv:2606.07555v1 Announce Type: cross Abstract: Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways. When this works, the

researcharxiv-cs-lg

RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks

arXiv:2606.07968v1 Announce Type: cross Abstract: Reasoning-capable large language models can be induced to spend their generation budget on injected decoy tasks rather than answering the user's quest

Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy

arXiv:2606.07929v1 Announce Type: new Abstract: Large language models (LLMs) are entering clinical practice based on benchmark accuracy that may fail to detect safety-relevant failure modes. Here we p

A Four-Condition Diagnostic Protocol for Evidence Utilization in Long-Context and Retrieval-Augmented Language Models

arXiv:2606.06758v1 Announce Type: new Abstract: Final-answer accuracy, retrieval recall, and citation overlap do not by themselves identify whether a long-context or retrieval-augmented language model

Korean Culture into LLM Alignment: Toward Cultural Coherence

arXiv:2606.06797v1 Announce Type: new Abstract: Cultural-aspect work on large language models is dominated by a negative target: which outputs to suppress. We argue that a constructive counterpart is

safetyarxiv-cs-cl

MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models

arXiv:2606.06696v1 Announce Type: cross Abstract: Vision and language models (VLMs) hold immense promise to transform biomedical imaging workflows, from detecting lesions in chest X-rays to profiling

Local Ai

The Geography of Algorithmic Judgment: LLM Intermediaries, Place Identity, and Racial Steering in Housing Search

arXiv:2606.06694v1 Announce Type: cross Abstract: Large language models (LLMs) are rapidly assuming an intermediary role in housing search through the integration of listing platforms within conversat

local-aiarxiv-cs-ai

The Geometry of Representational Failures in Vision Language Models

arXiv:2602.07025v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) exhibit puzzling failures in multi-object visual tasks, such as hallucinating non-existent elements or failing t

Industry

This is exactly right. People are starting to look for cheaper model alternatives and realizing two things at once: open-source models are a…

This is exactly right. People are starting to look for cheaper model alternatives and realizing two things at once: open-source models are already very good, and the ability to train and serve them ef

industryclem-delangue--x

Hardware

Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation

arXiv:2512.03086v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource progra

hardwarearxiv-cs-ai

Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

arXiv:2606.05792v1 Announce Type: new Abstract: TLA+ has supported industrial verification at companies such as Amazon and Microsoft, yet writing correct TLA+ specifications from natural language stil

Cross-Epoch Adaptive Rollout Optimization for RL Post-Training

arXiv:2606.05606v1 Announce Type: cross Abstract: LLM post-training often relies on reinforcement learning methods that sample multiple rollouts per prompt, yet most existing approaches use a fixed ro

DragOn: A Benchmark and Dataset for Drag-Based GUI Interactions

arXiv:2606.06322v1 Announce Type: new Abstract: GUI agents - vision-based models that control desktops, web browsers, and mobile devices through graphical user interfaces - promise to automate a wide

Goedel-Architect: Streamlining Formal Theorem Proving with Blueprint Generation and Refinement

arXiv:2606.06468v1 Announce Type: new Abstract: We introduce Goedel-Architect, an agentic framework for formal theorem proving in Lean 4 centered on blueprint generation and refinement. A blueprint is

QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

arXiv:2606.05875v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing ret

researcharxiv-cs-ai

Synthetic Contrastive Reasoning for Multi-Table Q&A

arXiv:2606.05382v1 Announce Type: new Abstract: Multi-table question answering requires models to retrieve relevant evidence, link schemas, and perform compositional reasoning across relational tables

Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs

arXiv:2602.09574v2 Announce Type: replace Abstract: Tree-search decoding is an effective form of test-time scaling for large language models (LLMs), but real-world deployment often imposes a fixed per

5 Jun 2026

ArcANE: Do Role-Playing Language Agents Stay in Character at the Right Time?

arXiv:2606.05553v1 Announce Type: new Abstract: Role-playing language agents (RPLAs) should play characters whose values and behavior evolve as the story progresses, not maintain a fixed persona. Exis