Search: “evaluation”

Predicting and Repairing Merge Collapse in Large Language Models

arXiv:2610.03199v1 Announce Type: cross Abstract: Large language models fine-tuned from a shared base can be merged by averaging their task vectors, but some merges collapse far below the base model,

researcharxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild

DGX agent

arXiv:2610.02413v1 Announce Type: new Abstract: LLM agents increasingly rely on activation probes as runtime monitors for prompt injection, jailbreaks, and unsafe requests, reading the model's own hid

model-releasesarxiv-cs-lg

5 Oct 2026

Rethinking RAG in Long Videos: What to Retrieve and How to Use It?

arXiv:2606.13141v2 Announce Type: replace Abstract: Retrieval-augmented generation is extending beyond text to long videos, where query-relevant chunks can be represented across multiple modalities an

Self-Supervised Scaling of Terminal Environments for Scientific Domains

arXiv:2610.02710v1 Announce Type: cross Abstract: Terminal agents are increasingly deployed beyond software engineering in science and other specialized domains. Constructing training environments req

agentsarxiv-cs-ai

Student-Guided Teacher Distillation for Efficient LLM Task Routing: Positioning Against Jev-Style System-1 Classifiers

arXiv:2610.02516v1 Announce Type: cross Abstract: Zero-shot classifiers are useful for routing user requests to specialized LLM tasks, but scoring every request against a large candidate set is expens

TerrainForge: Physics-Grounded road geometry Editing for Counterfactual Autonomous Driving

arXiv:2610.02825v1 Announce Type: new Abstract: Road geometry (e.g., crests, sags, and speed humps) and surface conditions (e.g., wet or icy pavement) affect how vehicles move, what drivers and onboar

safetyarxiv-cs-cv

TPBench: A Turning-Point Benchmark for Dialogue Compression

arXiv:2610.02736v1 Announce Type: cross Abstract: A compressor can keep the facts of a dialogue and still drop the turn that changed them. A user corrects a price, reverses a choice, or adds a constra

World Editing: Intervening on Executable Worlds at Increasing Depth

arXiv:2610.02331v1 Announce Type: new Abstract: Interactive world models are increasingly capable of generating environments and acting within them, yet deliberately editing an existing executable wor

Syntheses

Synthesis: Arxiv-Cs-Ai

synthesisarxiv-cs-aiauto-generated

Auto-generated synthesis of 29846 entries about arxiv-cs-ai

4 Oct 2026

Arex-2 vs swift vs qwen3.8 27b

Hi, Im gonna go all in on this, best finetune iv got my hands on for agentic coding, period. Test it yourself, im not gonna give you any benchmarks or evaluations, use your own tests and pracices, giv

agentsr-localllama

3 Oct 2026

Jev: Not Frontier, But Still Worth Your Attention [R]

model-releasesr-machinelearning

'TypeSafe AI sells Jev as a frontier-class reasoner that cannot hallucinate, built by the co-inventor of ChatGPT - fast, and almost free. We ran it live on 16,379 benchmark requests, measured its late

3 Oct 2026

A Deterministic and Auditable AI Security Risk Assessment Framework with ATLAS Aligned Executable Rules and Formal Verification

arXiv:2610.01436v1 Announce Type: new Abstract: Artificial intelligence systems are increasingly deployed in high impact and safety critical settings, yet security assessment remains difficult to repr

A First Glance at Jev for Network Traffic Classification: Accuracy, Processing Time, and Cost

model-releasesarxiv-cs-lg

arXiv:2610.00376v1 Announce Type: new Abstract: We evaluate Jev on ten dataset-defined application labels in CESNET-QUICEXT-25 using only the first ten packets' sizes, directions, and inter-packet tim

Are AI Coders Snitches? An Empirical Study of Pretraining Data Detection on Code Large Language Models

arXiv:2507.17389v2 Announce Type: replace-cross Abstract: Recent advances in code large language models (CodeLLMs) have made them indispensable tools in modern software engineering. However, these mod

Ask a Language Model for Lottery Numbers: Concentration in Repeated Six-of-49 Outputs

arXiv:2610.00052v1 Announce Type: cross Abstract: We evaluate six language-model configurations on requests for six distinct random integers from 1-49. Across 1,200 attempted calls using four English

researcharxiv-cs-ai

AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes

arXiv:2610.01861v1 Announce Type: new Abstract: Natural language descriptions can provide rich semantic representations of audio-visual urban scenes, yet datasets that jointly describe both auditory a

Beyond Final Accuracy: Auditing Communication in LLM Multi-Agent Systems

arXiv:2610.01042v1 Announce Type: new Abstract: Multi-agent communication aims to help agents benefit from one another's information. Yet improvements in system performance leave a fundamental ambigui

Can LLMs Reliably Annotate Bioassay Metadata to Improve Data Readiness?

arXiv:2610.01616v1 Announce Type: cross Abstract: The emergence of foundation models for molecular property prediction requires a high degree of AI data readiness, including reliable metadata annotati

researcharxiv-cs-ai

Closed-Loop Refinement and Execution for Learned Driving Planners

arXiv:2610.00992v1 Announce Type: cross Abstract: Learning-based driving planners are usually trained and evaluated in open loop against logged trajectories. In closed loop, a trajectory with small di

researcharxiv-cs-ro

Code That Works, Environments That Don't: Measuring Environment Reproducibility in AI-Generated Software

arXiv:2610.00425v1 Announce Type: cross Abstract: Code generation has emerged as a central capability of large language models, with coding agents now able to produce functionally correct software pro

agentsarxiv-cs-ai

Constitutional Value Potentials: reading and steering internal priority margins in language models

arXiv:2606.15420v2 Announce Type: replace-cross Abstract: Safety evaluations test a policy on prompts that omit the incentive information deployment supplies: a commission, a performance score, a dash

DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents

arXiv:2610.00316v1 Announce Type: cross Abstract: Voice agents enable low-latency, natural interaction, yet their ability to faithfully ground responses in external documents remains underexplored. We

Fault-Tolerant Budget Conservation in Distributed Multi-Agent Delegation

arXiv:2610.00349v1 Announce Type: new Abstract: Resource limits are becoming an authorization boundary for AI agents that delegate work across concurrent and failure-prone workers. Parent-child alloca

Local Ai

FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Training

arXiv:2610.01515v1 Announce Type: cross Abstract: Adaptive preconditioners accelerate model training, but heterogeneous client geometries can bias federated updates even when gradients are evaluated a

local-aiarxiv-cs-ai

Finding the Right Fit: Model-Harness Interactions across Agent Tasks

arXiv:2610.00917v1 Announce Type: new Abstract: Choosing an agent system means choosing both a language model and the harness through which it acts. We ask whether a strong model, harness, or pairing

FlashDexRetarget: Accelerating Dexterous Manipulation Data Generation through Multi-Motion Retargeting

model-releasesarxiv-cs-ro

arXiv:2610.01849v1 Announce Type: new Abstract: Human hand-object demonstrations offer a reusable source of dexterous robot manipulation data, but transferring them across embodiments requires physica

Greed Is Learned: Visible Incentives as Reward-Hacking Triggers

arXiv:2606.16914v2 Announce Type: replace Abstract: Safety evaluations test a policy on prompts that omit the incentive information deployment supplies: a commission, a performance score, a dashboard

Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents

arXiv:2610.00609v1 Announce Type: new Abstract: Legal research is a core and time-consuming legal workflow. Lawyers must identify controlling authority, verify that it remains valid, reconcile statute

Measuring Iterative Temporal Reasoning with Time Puzzles

arXiv:2601.07148v4 Announce Type: replace-cross Abstract: Tool use, such as web search, has become a standard capability even in freely available large language models (LLMs). However, existing benchm

Mimir: Physics-Grounded LLM Agents for Long-Horizon Irrigation Control

arXiv:2610.02038v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly combine reasoning, tool use, and action, but most evidence comes from episodic tasks with relatively imme

agentsarxiv-cs-ai

Multi-Party Backchannel Prediction: a Diagnosis, a Benchmark, and a Ceiling

arXiv:2610.01488v1 Announce Type: cross Abstract: Backchannel prediction has been studied almost entirely in dyadic conversation. We introduce a multi-party benchmark based on the AMI corpus, comprisi

Neural scaling laws and evolution of learnable activation functions of Kolmogorov-Arnold networks

model-releasesarxiv-cs-lg

arXiv:2610.00985v1 Announce Type: new Abstract: Kolmogorov-Arnold Networks (KANs) represent a compelling alternative to traditional Multi-Layer Perceptron (MLP)-based neural networks. By employing act

Robust Is Salient: An Informed Adversary Moves the Optimal Signal onto the Salience Pole

arXiv:2610.00233v1 Announce Type: new Abstract: When an informed adversary shares the audience of a constrained signalling channel, the signal that best protects the truth is the signal that best desc

researcharxiv-cs-ai

Scores That Hold, Benchmarks That Leak: Measuring Dataset Contamination in Public Brain-Tumor MRI Classification

arXiv:2610.00421v1 Announce Type: cross Abstract: Automated classification of brain tumors from MRI is a heavily published application of deep learning in medical imaging, with reported accuracies on

VoxelSynth3D: Interpretable Volumetric Image-Domain Metal Artifact Reduction with a Paired Synthetic CLINIC-Metal Benchmark

model-releasesarxiv-cs-cv

arXiv:2610.01512v1 Announce Type: new Abstract: Metal artifacts in postoperative musculoskeletal CT obscure bone-implant and adjacent soft-tissue interfaces. Many metal artifact reduction (MAR) method

WBAG: A Whole-Body and Attached-Geometry Safety Framework for Vision-Language-Action Manipulation

model-releasesarxiv-cs-ro

arXiv:2610.01083v1 Announce Type: new Abstract: Vision-language-action (VLA) policies have demonstrated impressive capabilities in generalizable robotic manipulation, but their deployment in the real

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I'v…

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks: Writing. Something I've had success with: Ask your LLM to explain something in ASD

researchkarpathy--x

Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities

arXiv:2610.00606v1 Announce Type: cross Abstract: Large Language Models increasingly serve as interfaces for knowledge-intensive information seeking tasks across languages by synthesizing multilingual

A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review

model-releasesarxiv-cs-cl

arXiv:2609.39027v1 Announce Type: new Abstract: AI reviewers can assign different judgments to manuscripts that report the same science in different wording, potentially rewarding rhetorical optimizat

Tutorials

AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization

arXiv:2609.39927v1 Announce Type: new Abstract: Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompts

tutorialsarxiv-cs-cl

Tutorials

Adaptive-GEPA: Make Your Harness Fit Heterogeneous Requests

arXiv:2609.38762v1 Announce Type: cross Abstract: Reflective optimizers such as GEPA improve language model prompts from execution traces and evaluator feedback; full-program extensions can also rewri

tutorialsarxiv-cs-ai

AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agents

arXiv:2609.40353v1 Announce Type: new Abstract: The task of 3D assembly requires translating an understanding of parts and their relationships into precise spatial arrangements. Can pretrained general

agentsarxiv-cs-cv

AutoDataBench: A Data-centric Testbed for Accelerating Auto Research

arXiv:2609.40097v1 Announce Type: new Abstract: Existing auto-research benchmarks often entangle multiple sources of improvement, including training frameworks, hyperparameters, compute budgets, and d

agentsarxiv-cs-cl

AVERT-VLN: Abstention-aware Visual Error Recovery and Training for Vision-and-Language Navigation

arXiv:2609.39579v1 Announce Type: new Abstract: Deploying vision-and-language navigation (VLN) agents in unseen environments remains challenging because unfamiliar layouts and visual conditions can ca

Benchmarking and Enhancing Skill-Level Memory for Partially Observable Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2609.38886v1 Announce Type: new Abstract: Recent advances in robot learning have enabled manipulation policies to perform increasingly diverse tasks and generalize across environments. However,

CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL

model-releasesarxiv-cs-cl

arXiv:2609.39533v1 Announce Type: new Abstract: During reinforcement learning with verifiable rewards (RLVR), large language models (LLMs) can exploit loopholes in their environments to obtain high re

Characterizing High Bandwidth Flash for LLM Serving

arXiv:2609.39131v1 Announce Type: new Abstract: Large language model (LLM) serving requires substantial memory to store model weights and KV caches. As models grow larger and contexts become longer, m

agentsarxiv-cs-lg

Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data

arXiv:2601.09717v2 Announce Type: replace-cross Abstract: Online medical consultations contain sensitive health information whose privacy implications depend not only on the entities mentioned but als