Search: “evaluation”

Cross-Cultural Bias in Mel-Scale Representations: Evidence and Alternatives from Speech and Music

arXiv:2604.10503v1 Announce Type: cross Abstract: Modern audio systems universally employ mel-scale representations derived from 1940s Western psychoacoustic studies, potentially encoding cultural bia

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

FPBench: A Comprehensive Benchmark of Multimodal Large Language Models for Fingerprint Analysis

DGX agent

arXiv:2512.18073v2 Announce Type: replace Abstract: Multimodal LLMs (MLLMs) are capable of performing complex data analysis, visual question answering, generation, and reasoning tasks. However, their

model-releasesarxiv-cs-cv

14 Apr 2026

PaperScope: A Multi-Modal Multi-Document Benchmark for Agentic Deep Research Across Massive Scientific Papers

arXiv:2604.11307v1 Announce Type: new Abstract: Leveraging Multi-modal Large Language Models (MLLMs) to accelerate frontier scientific research is promising, yet how to rigorously evaluate such system

Relational Preference Encoding in Looped Transformer Internal States

arXiv:2604.09870v1 Announce Type: cross Abstract: We investigate how looped transformers encode human preference in their internal iteration states. Using Ouro-2.6B-Thinking, a 2.6B-parameter looped t

Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation

arXiv:2604.10720v1 Announce Type: new Abstract: Artificial models that simulate how learners act and respond within educational systems are a promising tool for evaluating tutoring strategies and feed

How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms

arXiv:2604.08966v1 Announce Type: new Abstract: While Multimodal Large Language Models (MLLMs) have advanced Video Temporal Grounding (VTG), existing methods often couple output paradigms with differe

13 Apr 2026

AI-Driven Research for Databases

arXiv:2604.06566v1 Announce Type: cross Abstract: As the complexity of modern workloads and hardware increasingly outpaces human research and engineering capacity, existing methods for database perfor

Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding

arXiv:2603.18472v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) perform strongly on natural images, yet their ability to understand discrete visual symbols remains u

Applications

Large Language Models for Outpatient Referral: Problem Definition, Benchmarking and Challenges

arXiv:2503.08292v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly applied to outpatient referral tasks across healthcare systems. However, there is a lack of stan

applicationsarxiv-cs-ai

SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glasses

arXiv:2602.22683v2 Announce Type: replace Abstract: The rapid advancement of AI-powered smart glasses-one of the hottest wearable devices-has unlocked new frontiers for multimodal interaction, with Vi

Behavioral Forgetting Is Not Privacy in Machine Unlearning

arXiv:2610.21799v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted information from large language models, but existing evaluations often measure forgetting only on a designate

CriticaLLM: Using LLMs as Autonomous Game Master for Tabletop Role-Playing Games

arXiv:2610.21525v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown impressive capabilities across many tasks, including processing large amounts of text, following instructions, a

Decoding What iOS Location Services Send to Apple: An LLM Is an Unreliable Analyst but a Useful Toolmaker

arXiv:2610.21597v1 Announce Type: cross Abstract: Reverse engineering the data formats used by mobile software is skilled and slow, and using AI to assist with it is attractive. We evaluate this for t

From Perception to Prediction and Intervention: Benchmarking Proactive Video Assistants in Real-Life Scenarios

model-releasesarxiv-cs-lg

arXiv:2610.21568v1 Announce Type: new Abstract: Recent multimodal large language models have advanced video understanding, but existing benchmarks mainly evaluate whether models can interpret observed

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

arXiv:2610.20776v1 Announce Type: cross Abstract: LLM-based agents increasingly execute long-horizon, end-to-end workflows involving shell commands, code modification, dependency installation, and int

LOADBench: A Large Open Analog Dataset and Benchmark for Machine Learning in IC Design

arXiv:2610.21591v1 Announce Type: new Abstract: Machine learning (ML) for analog IC design is constrained by limited open data and the absence of standardized evaluation: results reported in different

Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?

arXiv:2610.22155v1 Announce Type: new Abstract: Real-time interaction requires agents to perceive evolving environments and act under temporal constraints, yet existing evaluations of real-time stream

researcharxiv-cs-ai

RLDF: A Region-Level Deepfake Dataset and Benchmark for Partial-Face Manipulation Detection

arXiv:2610.21438v1 Announce Type: new Abstract: Existing deepfake detection models are built upon manipu- lated images that are tampered with either exclusively fully swapped or fully synthesised face

Adaptive-Group Process Rewards for GUI Agents

arXiv:2610.17710v1 Announce Type: new Abstract: Training graphical user interface (GUI) agents with reinforcement learning suffers from sparse outcome rewards, leading to inefficient learning. Process

Aligning Faithfulness for Theory-Scale Autoformalization

arXiv:2610.18498v1 Announce Type: cross Abstract: Autoformalization (AF) requires more than compilation: formal statements must preserve the meaning of their informal sources, a property known as fait

Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study

arXiv:2505.21710v2 Announce Type: replace Abstract: This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate

Hardware

DEEPRUBRIC: Evidence-Tree Rubric Supervision for Efficient Reinforcement Learning of Deep Research Agents

arXiv:2606.17029v2 Announce Type: replace Abstract: Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewar

hardwarearxiv-cs-cl

DynThinker: Unifying Static 3D and Dynamic 4D Spatial Reasoning

arXiv:2610.20598v1 Announce Type: new Abstract: Achieving general spatial intelligence requires a unified understanding of both static 3D and dynamic 4D. However, we find that existing models struggle

Edgewise Envelopes Between Balanced Forman and Ollivier-Ricci Curvature

arXiv:2603.13535v5 Announce Type: cross Abstract: Evaluating Ollivier-Ricci (OR) curvature on large-scale graphs is computationally prohibitive due to the necessity of solving an optimal transport pro

researcharxiv-cs-lg

Fully sequential Gaussian process regression for preference learning

arXiv:2610.19391v1 Announce Type: new Abstract: We study continuous global optimization problems in which the objective function cannot be directly evaluated; rather, we observe only which of two sele

safetyarxiv-cs-lg

Heterogeneous Agentic Coevolution

arXiv:2610.16078v1 Announce Type: new Abstract: Recent advances in self-improving LLM agents have demonstrated the potential of AI systems that drive their own intelligence. However, many domain-speci

HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks

arXiv:2610.16319v1 Announce Type: new Abstract: Real-world hybrid web tasks need both information gathering and page interaction, yet search-based and operation-based agents are developed and evaluate

More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning

arXiv:2610.19789v1 Announce Type: new Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseli

Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving

arXiv:2503.21821v2 Announce Type: replace Abstract: We introduce Physics, a benchmark for evaluating foundation models on advanced physics problem solving. It contains 1034 annotated, open-ended probl

ProCARE: Real-World Study Automation via Profile-Grounded Evidence Contracts

arXiv:2610.16175v1 Announce Type: new Abstract: Real-world studies (RWS) analyze routinely collected healthcare data, such as electronic health records and insurance claims, to estimate treatment effe

PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

arXiv:2606.18060v2 Announce Type: replace Abstract: As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Oth

PXRD-Bench: How Close Are We to Determining Crystal Structures from Powder Diffraction?

arXiv:2610.16480v1 Announce Type: new Abstract: Determining crystal structures from powder X-ray diffraction (PXRD) would enable routine characterization of newly synthesized materials, including thos

Local Ai

RA^2: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning

arXiv:2610.19385v1 Announce Type: new Abstract: Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failur

local-aiarxiv-cs-lg

Recognizing Dialogue Ambiguity via Candidate Set Visual Grounding

arXiv:2610.19723v1 Announce Type: new Abstract: Interpreting an ambiguous utterance in real-world interactions often requires context beyond a single utterance. When the utterance refers to objects in

Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support

arXiv:2609.32817v2 Announce Type: replace Abstract: Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

arXiv:2610.16814v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as pa

The Anatomy of a Blind Spot: What an LLM Judge Must Be Shown, and How Much

arXiv:2610.15938v1 Announce Type: new Abstract: Evaluation of text generated by large language models (LLMs) is increasingly delegated to LLMs acting as judges, yet what such a judge must be shown rem

researcharxiv-cs-ai

The Silent Failures in Agentic Data Science

arXiv:2610.14115v1 Announce Type: new Abstract: Large language models have significantly lowered the barrier to writing code. Naturally, this has also led to the development of data science agents tha

Towards a Benchmark for Fire Safety Code Interpretation in the Hong Kong AEC Industry

arXiv:2610.16127v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly explored in the Architecture, Engineering, and Construction (AEC) domain, yet their reliability in regul

Upstream Influence, Downstream Re-Anchoring in LLM Agent Chains

arXiv:2610.17723v1 Announce Type: new Abstract: People act on evaluative judgments produced by LLM agents, and in multi-agent systems those judgments pass through several agents before reaching a pers

When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO

arXiv:2610.20089v1 Announce Type: new Abstract: Pairwise preference feedback has become a widely adopted supervision signal for reinforcement learning in video generation models, as pairwise compariso

safetyarxiv-cs-cv

A CaLLM Judge is a Better Judge: Mitigating Bias in LLM-as-a-judge

arXiv:2610.18108v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as judges to predict which of two candidate answers a human evaluator would prefer, providing a scala

safetyarxiv-cs-lg

APT: Feedback-Driven Agentic Post-Training for Text-to-Image Generation

arXiv:2610.18097v1 Announce Type: new Abstract: Post-training is crucial for improving text-to-image models, yet existing pipelines typically rely on manually curated training data and fixed training

CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT

arXiv:2610.14226v1 Announce Type: new Abstract: Purpose : To develop and evaluate CAPTOR, a deep-learning vision-language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. M

researcharxiv-cs-cv

CESNET-TimeSeries26: A Multi-Year Network Traffic Time Series Dataset for Annual Seasonality, Concept Drift, and Long-Term Anomaly Detection

model-releasesarxiv-cs-lg

arXiv:2610.14799v1 Announce Type: cross Abstract: Forecast-based models are fundamental to network traffic management and anomaly detection. However, it is essential to evaluate these forecasting mode

CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

arXiv:2610.16577v1 Announce Type: new Abstract: Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes pro

PerturbBridge: Direct Phenotypic Supervision for Cellular Perturbation Generation

arXiv:2610.17066v1 Announce Type: new Abstract: Virtual cell models should capture phenotypes induced by specific perturbations. However, image-based models are typically trained and evaluated to matc

researcharxiv-cs-cv

PLDiv: Measuring Dataset Diversity and Semantic Collapse via Persistence Landscapes

arXiv:2610.14031v1 Announce Type: cross Abstract: Modern data pipelines increasingly combine human-created and human-curated data with AI-generated data for model training and evaluation. As datasets

researcharxiv-cs-cv