Search: “evaluation”

Pretraining Language Models on Historical Text

arXiv:2606.02991v1 Announce Type: cross Abstract: We introduce TypewriterLM, a 7.24B History language model (LM) trained exclusively on English text predating 1913. Developing History LMs requires add

Reasoning Structure of Large Language Models

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2606.03883v1 Announce Type: new Abstract: Large reasoning models (LRMs) are often evaluated using metrics such as final-answer accuracy or token count. However, identical scores on these metrics

model-releasesarxiv-cs-ai

3 Jun 2026

RESCAST-100K: A Comprehensive Dataset for Cross-Domain Residential Load and Indoor Temperature Forecasting

model-releasesarxiv-cs-lg

arXiv:2606.02852v1 Announce Type: new Abstract: Accurate short-term forecasting of residential energy load and indoor temperature is essential for home energy management systems, grid-level demand res

Rethinking Molecular Text Representations for LLMs: An Empirical Study

arXiv:2606.03057v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for molecular tasks, but it remains unclear which molecular representation to use. We present a sys

scTranslation: A Comprehensive Benchmark for Single-Cell Multi-Omics Modality Translation

arXiv:2606.03906v1 Announce Type: new Abstract: Simultaneous measurement of multiple omics modalities in single cells enables researchers to gain a more comprehensive understanding of cellular states

Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection

arXiv:2606.02601v1 Announce Type: new Abstract: Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection. We show that this protoco

researcharxiv-cs-lg

Agents

The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios

arXiv:2601.08173v2 Announce Type: replace Abstract: The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets perform

agentsarxiv-cs-ai

The Unsampled Truth: Psychometrics in SLMs Measure Prompt Artifacts, Not Psychological Constructs

arXiv:2606.03357v1 Announce Type: cross Abstract: When prompting SLMs for psychometric assessments, researchers assume the outputs reflect semantic reasoning. We evaluate this premise across 13 open-w

researcharxiv-cs-ai

Towards a Science of AI Agent Reliability

arXiv:2602.16666v3 Announce Type: replace Abstract: AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many age

TriEval: A Resource-Efficient Pipeline for LLM Bias, Toxicity, and Truthfulness Assessment

arXiv:2606.03036v1 Announce Type: new Abstract: LLMs have evolved from basic chatbots to the backbone of the AI ecosystem, now widely used in healthcare, schools, and government services. The domain-w

VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearch

arXiv:2606.03273v1 Announce Type: cross Abstract: Visual DeepSearch requires multimodal large reasoning model (MLRM) agents to answer complex visual queries by repeatedly inspecting image regions, gro

A Fiber Criterion for Representation Identifiability in Supervised Learning

arXiv:2606.01092v1 Announce Type: cross Abstract: Supervised learning evaluates predictors through their input-output behavior. When a predictor is implemented as a composition f=circ h, supervised ev

researcharxiv-cs-ai

A Methodological Framework for Explicit Control of the Speed-Accuracy Trade-off in Brain-Computer Interfaces

arXiv:2606.00106v1 Announce Type: cross Abstract: Brain-computer interfaces (BCIs) are limited by low signal-to-noise ratio in modalities such as electroencephalography, which requires multiple trials

Acoustic and perceptual differences between standard and accented speech and their voice clones

arXiv:2604.01562v2 Announce Type: replace-cross Abstract: Voice cloning is often evaluated in terms of overall quality, but less is known about accent preservation and its perceptual consequences. We

researcharxiv-cs-ai

Adaptive Dense Evidence Refinement for Video Relational Reasoning for VRR-QA Challenge

arXiv:2606.01104v1 Announce Type: new Abstract: VRR-QA evaluates whether video-language systems can infer spatial, temporal, viewpoint, depth, and visibility relations that are not always resolved by

researcharxiv-cs-cv

Before and After Temperature: A Distributional View of Creative LLM Generation

arXiv:2606.01451v1 Announce Type: new Abstract: Reference-free evaluation of large language model (LLM) creativity relies on perplexity, entropy, and top-1 margin. We show that a much stronger signal

Benchmarking Local LLMs for Natural-Language-to-SQL Querying in Biopharmaceutical Manufacturing: An Empirical Benchmark on Consumer-Grade Hardware

arXiv:2606.01338v1 Announce Type: new Abstract: Biopharmaceutical manufacturing organizations operate under regulatory frameworks such as FDA guidance, EU Good Manufacturing Practice (GMP), and the EU

Beyond Rigid: Benchmarking Non-Rigid Video Editing

arXiv:2601.18340v2 Announce Type: replace Abstract: As video generation models are increasingly expected to manipulate physical dynamics, there is a growing need to move evaluation beyond appearance f

Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy

arXiv:2606.00065v1 Announce Type: cross Abstract: Automated extraction of materials composition-property data from scientific literature has advanced considerably with the development of large languag

Tutorials

Beyond Tool Adoption: A Practical Five-Stage Developmental Continuum for AI Literacy in Higher Education

arXiv:2606.00038v1 Announce Type: cross Abstract: Artificial intelligence (AI) literacy is increasingly recognized as a foundational competency for all university graduates. Yet students' engagement w

tutorialsarxiv-cs-ai

Business Utility of Large Language Models as Exploratory Data Analysis Agents

arXiv:2606.00051v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in analytical workflows, but their suitability as exploratory data analysis (EDA) agents in busines

Can LLMs Reason Structurally? Benchmarking via the Lens of Data Structures

arXiv:2505.24069v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are deployed on increasingly complex tasks that require multi-step decision-making. Understanding their algorithm

CASTLE2026 Team WDL Technical Report

arXiv:2606.00712v1 Announce Type: new Abstract: The CASTLE Challenge @ EgoVis 2026 evaluates long-form egocentric video question answering over 600+ hours of multi-perspective recordings. Each four-ch

CityTrajBench: A Unified Benchmark for City-Scale Vehicle Trajectory Generation

arXiv:2606.02287v1 Announce Type: cross Abstract: Urban trajectory generation is a fundamental task for transportation simulation, urban planning, and mobility analytics. However, systematic compariso

COMAP: Co-Evolving World Models and Agent Policies for LLM Agents

arXiv:2606.02372v1 Announce Type: new Abstract: Equipping language agents with world models enables them to anticipate environment dynamics and evaluate candidate actions before execution. However, ex

Applications

Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument

arXiv:2606.01643v1 Announce Type: new Abstract: Sign Language Production (SLP) is the task of generating avatar sign language motion from natural language text. The quality of the generated motion is

applicationsarxiv-cs-cv

Context-aware child-directed speech detection from long-form recordings

arXiv:2606.01134v1 Announce Type: cross Abstract: Automatically distinguishing child-directed speech from adult-directed speech in long-form recordings is key to scalable analyses of children's langua

researcharxiv-cs-lg

Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking

arXiv:2606.01120v1 Announce Type: new Abstract: In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence. Their parametric knowledge can ind

Agents

Ev-Trust: An Evolutionarily Stable Trust Mechanism for Decentralized LLM-Based Multi-Agent Service Economies

arXiv:2512.16167v3 Announce Type: replace-cross Abstract: Decentralized LLM-based multi-agent service economies face three vulnerabilities that undermine traditional trust mechanisms: reduced cost of

agentsarxiv-cs-ai

Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification

arXiv:2606.01160v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used with formal interactive theorem provers such as Lean 4. Scaling these systems with reinforcement lear

Failure of contextual invariance in large language models

arXiv:2603.23485v2 Announce Type: replace-cross Abstract: Standard evaluation practices assume that large language model (LLM) outputs are stable when prompts are embedded in contextually equivalent d

French parsing enhanced with a word clustering method based on a syntactic lexicon

arXiv:2606.00634v1 Announce Type: new Abstract: This article evaluates the integration of data extracted from a French syntactic lexicon, the Lexicon-Grammar (Gross, 1994), into a probabilistic parser

researcharxiv-cs-cl

InsightVQA: High-Dimensional Emotion-Cognitive Visual Question Answering Benchmark

arXiv:2606.02171v1 Announce Type: new Abstract: Visual emotion understanding requires models not only to recognize emotional states, but also to why they arise and perform higher-level cognitive reaso

Interpretable Policy Distillation for Power Grid Topology Control

arXiv:2606.00561v1 Announce Type: cross Abstract: Deep reinforcement learning (RL) offers a promising route to real-time power grid operation, yet large neural policies are costly to evaluate, hard to

K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts

arXiv:2606.02404v1 Announce Type: new Abstract: Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, b

LLM4Cov: Execution-Aware Agentic Learning for High-coverage Testbench Generation

arXiv:2602.16953v3 Announce Type: replace Abstract: Execution-aware LLM agents offer a promising paradigm for learning from tool feedback, but such feedback can be expensive and slow to obtain, making

Markerless Augmented Reality Registration for Surgical Guidance: A Multi-Anatomy Clinical Accuracy Study

arXiv:2511.02086v2 Announce Type: replace Abstract: Purpose: In this paper, we develop and clinically evaluate a depth-only, markerless augmented reality (AR) registration pipeline on a head-mounted d

safetyarxiv-cs-cv

MASCOT: Towards Multi-Agent Socio-Collaborative Companion Systems

arXiv:2601.14230v2 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) are emerging as promising socio-collaborative companions for emotional and cognitive support. However, existing syst

MBench: A Comprehensive Benchmark on Memory Capability for Video World Models

arXiv:2606.00793v1 Announce Type: new Abstract: Recent advancements in video-based world models have demonstrated an unprecedented ability to synthesize high-fidelity visual sequences. However, a fund

Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning

arXiv:2606.01301v1 Announce Type: new Abstract: Hallucinations in medical large language models (LLMs) pose serious risks for clinical decision support, particularly when models must reason over compl

MedGym:A Unified Continuous-Time Benchmark for Dynamic Medical Treatment Reinforcement Learning

model-releasesarxiv-cs-lg

arXiv:2606.01028v1 Announce Type: new Abstract: Medical treatment recommendation poses several challenges to reinforcement learning (RL): patient physiology evolves in continuous time, measurements an

MemoNoveltyAgent: A Historical Research Memory-Aware Agent Workflow for Paper Novelty Assessment

arXiv:2603.20884v2 Announce Type: replace Abstract: To alleviate the heavy burden of paper screening, researchers increasingly rely on existing AI agents, such as AI reviewers or DeepResearch, for pap

Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation

arXiv:2509.03456v2 Announce Type: replace-cross Abstract: Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits. Recent advances

safetyarxiv-cs-lg

Applications

Optimal Transport-based Permutation-Invariant Bayesian Optimization of Offshore Wind Farm Layouts

arXiv:2606.00009v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely and successfully adopted for solving optimization problems having an expensive-to-evaluate, black-box, and non-conv

applicationsarxiv-cs-ai

POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems

arXiv:2606.02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and halluci

Position: Good Embodied Reward Models Need Bad Behavior Data

arXiv:2606.01036v1 Announce Type: new Abstract: This position paper argues that to obtain reliable embodied reward models, the community must invest in ``bad'' robot data: failed, suboptimal, error-pr

safetyarxiv-cs-ro

Question-Aware Evidence Ledgers for Video Relational Reasoning

arXiv:2606.02506v1 Announce Type: new Abstract: The VRR-QA challenge evaluates visual relational reasoning in videos, where answers often depend on implicit spatial relations, event boundaries, target

R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking

arXiv:2606.01113v1 Announce Type: new Abstract: The CoVR-R challenge evaluates composed video retrieval, where a system must retrieve a target video from a large gallery given a reference video and a

researcharxiv-cs-cv