Search: “evaluation”

ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring

arXiv:2610.06744v1 Announce Type: cross Abstract: ufakzeka-karar is an open Turkish decision model with 182,494,466 parameters. Given a Turkish text and questions of a fixed answer type (a choice, a l

6 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos

DGX agent

arXiv:2610.04432v1 Announce Type: new Abstract: Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manu

model-releasesarxiv-cs-cv

6 Oct 2026

What Will Post-Training Fix? Per-Problem Gains Are Shared Across Independent RL Runs, and Existing Checkpoints Predict Them Better Than A Priori Signals

arXiv:2610.04978v1 Announce Type: new Abstract: Data selection, curricula and the evaluation of post-training recipes all assume that we can tell, before training, which problems a model will improve

6 Oct 2026

When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models

arXiv:2610.05550v1 Announce Type: new Abstract: The component that dominates a latent world model's prediction error need not be the one whose repair most improves action selection. We show this by co

researcharxiv-cs-lg

6 Oct 2026

A Benchmark for Spatially Grounded Gesture Generation

arXiv:2610.03105v1 Announce Type: new Abstract: Communication in shared space interweaves verbal and non-verbal signals, and pointing gestures anchor language to the environment: 'put the cup on that

A Controlled Audit of Personal AI Memory for Rating Prediction

arXiv:2610.02764v1 Announce Type: new Abstract: In structured rating prediction, does a personal AI use historical item-rating associations, or mainly the user's rating tendencies? We audit this disti

Can AI Understand the Language of Origami?

arXiv:2603.13856v3 Announce Type: replace-cross Abstract: Building AI systems that can plan, act, and create in the physical world requires more than pattern recognition. Such systems must reason abou

Capability Scaling-Down Laws for LLM Compression

arXiv:2610.02462v1 Announce Type: cross Abstract: LLM compression reduces inference costs and memory requirements, but selecting a method and configuration remains largely empirical because comparable

researcharxiv-cs-cl

Clinical Concept Centers in LLMs

arXiv:2610.02829v1 Announce Type: new Abstract: Large language models are increasingly used in clinical settings. However, research into the reliability and performance of these models has focused alm

researcharxiv-cs-cl

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs

arXiv:2610.02657v1 Announce Type: new Abstract: Converting a pretrained autoregressive (AR) model to a diffusion language model (dLLM) enables parallel generation without pretraining a new model. Publ

ConvoDrift: A Multi-Turn Conversational Dataset for Modeling Stylistic Tone Evolution

arXiv:2610.02873v1 Announce Type: cross Abstract: The evolution of linguistic style in conversations is an underexplored issue in NLP. Most style-control datasets focus on sentences or assume a static

safetyarxiv-cs-ai

Cross-cohort TB classification using clinical data gathered in Uganda and South Africa

arXiv:2610.03256v1 Announce Type: new Abstract: We present a first evaluation of machine learning applied to patient clinical and demographic data gathered in two different countries for the purpose o

researcharxiv-cs-lg

CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking

arXiv:2610.02460v1 Announce Type: new Abstract: Recent benchmarks rely on user simulators to evaluate AI agents in multi-turn interaction. While existing simulation techniques demonstrate surface fide

safetyarxiv-cs-cl

CVE2AP: Automated Generation of PDDL-Encoded Attack Paths via Large Language Models

arXiv:2610.03383v1 Announce Type: new Abstract: Attack Path (AP) modeling is fundamental to cybersecurity analysis, where the Planning Domain Definition Language (PDDL) has been widely adopted to enco

DeltaWorld: Physically Consistent Interactive World Simulators via Action-Conditioned Latent Increment Learning

model-releasesarxiv-cs-ro

arXiv:2610.02691v1 Announce Type: new Abstract: Interactive world simulators can provide scalable environments for robot planning, policy training, and evaluation by predicting action consequences whi

EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents

arXiv:2610.03153v1 Announce Type: cross Abstract: Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resources

MiDShip: Multimodal Dataset of Ship Cargo Hold Structures for Engineering Design

arXiv:2610.02214v1 Announce Type: cross Abstract: Ship structures govern vessel strength, safety, and manufacturability, but their design must satisfy hundreds of classification society requirements,

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

model-releasesarxiv-cs-cl

arXiv:2610.02986v1 Announce Type: new Abstract: Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without acces

Output Language Confusion under Multilingual Prompt Contamination

model-releasesarxiv-cs-cl

arXiv:2610.02926v1 Announce Type: new Abstract: Standard factual benchmarks assume clean monolingual prompts and exact-match scoring, two assumptions that break simultaneously in real-world multilingu

Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces

arXiv:2610.02580v1 Announce Type: new Abstract: Physical AI Smart Spaces is, to the best of our knowledge, the first benchmark to simultaneously provide large-scale, multi-class, and multi-camera 3D p

PowerBench: Measuring Language Model Bias in Power-shifting Requests

arXiv:2610.02303v1 Announce Type: new Abstract: Language models increasingly assist people with power-related requests, so systematic differences in whom they help could shift the distribution of powe

safetyarxiv-cs-lg

Rank-Aware Speculative Sampling for Diffusion Draft Trees

arXiv:2610.02251v1 Announce Type: new Abstract: Speculative sampling accelerates diffusion generation by verifying inexpensive draft states in parallel while preserving the target law. Recent tree-bas

researcharxiv-cs-lg

StanceEval 2026: The Second Stance Detection Shared Task

arXiv:2610.03215v1 Announce Type: new Abstract: StanceEval 2026 is the second edition of the StanceEval shared task series on stance detection in Arabic social media text. Stance detection aims to ide

researcharxiv-cs-cl

TasteBench: Multimodal Benchmark for Sensory Prediction, from Molecules to Sustainable Foods

arXiv:2610.02599v1 Announce Type: new Abstract: Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug an

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

arXiv:2610.03585v1 Announce Type: cross Abstract: Security benchmarks for LLM-based agents often report the attack success rate (ASR) as a measure of model robustness and use these scores to compare d

World Embedding Benchmark

model-releasesarxiv-cs-cl

arXiv:2610.03632v1 Announce Type: cross Abstract: Physical fidelity has received increasing attention in world models and video generation, yet how video representations encode physical information re

Local Ai

Writerslogic at PAN 2026: Process over Content for Robust Detection under Domain Shift

arXiv:2610.03565v1 Announce Type: new Abstract: We describe the Writerslogic systems for three PAN at CLEF 2026 shared tasks (Reasoning Trajectory Detection, Voight-Kampff Generative AI Detection, and

local-aiarxiv-cs-cl

Benchmarking Generative Models for Weather Data Assimilation on Real Station Observations

arXiv:2610.00728v1 Announce Type: cross Abstract: Weather reanalysis products rely on computationally intensive numerical weather predictions followed by data assimilation that corrects the forecast t

CAGE-NAS: Certified Functional Descent for Efficient Model Growth

arXiv:2610.01173v1 Announce Type: new Abstract: The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be ex

CHORDONOMICON: A Dataset of 666,000 Songs and their Chord Progressions

arXiv:2410.22046v4 Announce Type: replace-cross Abstract: Chord progressions encapsulate important information about music, pertaining to its structure and conveyed emotions. They serve as the backbon

Conflicting Supervision Moves Commitment, Not Capability: A 12.29{sigma} arrangement effect that is exactly zero under a convention-agnostic score

arXiv:2610.00234v1 Announce Type: new Abstract: 'Train a model on the same problems written under two incompatible conventions, both correct, and ask what the ordering of that data writes into the par

Crossing the Cyber Divide: Sim-to-Sim and Sim-to-Real Transfer for RL Agents

arXiv:2610.00759v1 Announce Type: cross Abstract: Cyber attack agents are typically trained and evaluated within a single simulator, making it unclear whether learned policies transfer beyond the envi

safetyarxiv-cs-lg

Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability

arXiv:2610.01168v1 Announce Type: cross Abstract: Time Series Anomaly Detection has received increasing attention, driven by the growing availability of complex time series data. This surge has led to

Does Continual Imitation Learning Remain Grounded? A Language-Perturbed Benchmark for Robotic Task Retention

model-releasesarxiv-cs-ro

arXiv:2610.00542v1 Announce Type: new Abstract: Continual imitation learning evaluates whether a robot can learn new knowledge without forgetting previously learned skills. However, retaining task per

Agents

From Reasoning Failures to Composable Video Spatial Intelligence

arXiv:2610.01999v1 Announce Type: new Abstract: Spatial reasoning benchmarks evaluate vision-language models across diverse tasks, but task-level scores do not reveal which underlying capabilities acc

agentsarxiv-cs-cv

GPEC: Efficient Pre-LLM Gaussian Process Embedding Correction for Cardiac Video Caption Generation

arXiv:2610.00196v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown strong potential for video understanding and caption generation, but their performance may decline i

How Much Can Language Models Gain from Test-Time Computation?

arXiv:2610.01110v1 Announce Type: new Abstract: How much can test-time computation improve a language model, and at what cost? Test-time scaling is widely proposed as a substitute for larger models, b

MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation

arXiv:2610.00604v1 Announce Type: cross Abstract: Vision-language-action policies often see only one or a few recent frames, which makes it difficult to evaluate how they use information that disappea

Mingbird: A Local-First Agent Harness Enabling Small Open Models to Complete Real Tasks

arXiv:2610.02001v1 Announce Type: new Abstract: Small open-weight models (2-9B) run on ordinary laptops, but under cloud-scale agent harnesses they rarely complete real tasks: tool prefill overflows t

PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents

arXiv:2610.01349v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents turn generated text into real side effects, so poisoned tool metadata, retrieved pages, memory, and reusa

RIQE: a NIQE-style reference model for Computed Tomography

arXiv:2610.00384v1 Announce Type: cross Abstract: The Natural Image Quality Evaluator (NIQE) scores an image by its statistical distance from a model fitted on pristine images, and its distributed mod

researcharxiv-cs-cv

ROWBench: Do Video Models Render What the Program Specifies?

arXiv:2610.02205v1 Announce Type: new Abstract: Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However

Scalable Multi-Task Inverse Reinforcement Learning

arXiv:2610.00758v1 Announce Type: new Abstract: By learning transferable rewards, inverse reinforcement learning (IRL) enables counterfactual evaluation of agents under modified environments. Such tra

safetyarxiv-cs-lg

The Geometry of Time: Horizon-Independent Feasibility and Repair for STL

arXiv:2610.00199v1 Announce Type: cross Abstract: Signal Temporal Logic control synthesis frequently encounters physical infeasibility due to actuator limits or flawed task deadlines. Standard optimiz

researcharxiv-cs-ro

TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models

arXiv:2610.00835v1 Announce Type: new Abstract: Text-to-music generation models are trained on massive music collections, creating a growing need for data attribution methods that can quantify the con

Local Ai

Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning

arXiv:2610.02190v1 Announce Type: new Abstract: Step-size selection remains a central challenge in large-scale neural network optimization; conservative steps slow convergence, while aggressive steps

local-aiarxiv-cs-lg

VideoSTF: Stress-Testing Output Repetition in Video Large Language Models

arXiv:2602.10639v2 Announce Type: replace Abstract: Video Large Language Models (VideoLLMs) have achieved strong performance on video understanding tasks, yet existing benchmarks evaluate only what mo

researcharxiv-cs-cv

VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision

arXiv:2610.00666v1 Announce Type: cross Abstract: Qualitative comparison figures are central evidence in computer vision papers, and vision-language models (VLMs) are increasingly used to judge them.