Search: “evaluation”

Synth-JDoc: Synthesizing a Japanese Document Image Dataset for OCR with Diverse Layouts and Embedded Images

arXiv:2608.28248v1 Announce Type: cross Abstract: The ability of Large Vision Language Models (LVLMs) to read text within document images is crucial, as it enables various applications such as Documen

31 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

What Makes Agent Memory Useful for Reliable Unanswerable Question Handling?

DGX agent

arXiv:2608.27924v1 Announce Type: new Abstract: Reliable handling of unanswerable questions (UAQs) is critical for trustworthy LLM-based agents. Although memory is widely used in agent systems, its ro

agentsarxiv-cs-cl

31 Aug 2026

When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI

arXiv:2608.28518v1 Announce Type: cross Abstract: We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that

safetyarxiv-cs-cl

31 Aug 2026

Tutorials

1/ An equally consistent description is: Multiple model instances encountered persistent shared state, inherited tools and discoveries from …

1/ An equally consistent description is: Multiple model instances encountered persistent shared state, inherited tools and discoveries from earlier runs, and optimized against common evaluation incent

tutorialsyann-lecun--x

30 Aug 2026

A Task-Centric Ontology and Deterministic Domain Rules as a Verifiable Core for AI-Assisted Chemistry Problem Solving

arXiv:2608.26164v1 Announce Type: new Abstract: Large language models can interpret natural-language chemistry questions, but their internal reasoning is difficult to inspect, constrain, and validate.

researcharxiv-cs-ai

Accelerating Scientific Research with Gemini in the Real-World

arXiv:2608.26701v1 Announce Type: new Abstract: We present an extension and comprehensive real-world validation of Co-Scientist, a Gemini-based multi-agent system designed to accelerate end-to-end sci

AEScorer: An Agentic Evidence-Grounded Framework for Graded Factuality Verification

arXiv:2601.03605v2 Announce Type: replace Abstract: Despite the significant advancements of Large Language Models (LLMs), their factuality remains a critical challenge, creating a growing need for mor

Autoresearch with Coding Agents: Generalizers and Metric-Maximizers on Quran Recitation Data

arXiv:2607.18064v2 Announce Type: replace-cross Abstract: Coding agents can now be left alone to improve software against a score. In this pattern--recently popularized as 'autoresearch'--the agent re

BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click

arXiv:2608.26867v1 Announce Type: new Abstract: Large language model agents reason, call tools, and act autonomously over many steps, but their agentic skills-correctly sequencing tools, planning unde

Benchmarking AI Agents for Hardware Design Automation via MCP Tool Calling

arXiv:2608.26199v1 Announce Type: new Abstract: We ask whether AI agents powered by locally deployed large language models can reliably automate expert-defined hardware design workflows in an industry

BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Models

arXiv:2608.27268v1 Announce Type: new Abstract: Although Large language models (LLMs) mediate access to knowledge and computational assistance, their capabilities should benefit vulnerable groups in t

Clinically Aligned Geometry Constraints for Robust IVUS Vessel Boundary Segmentation

arXiv:2606.18723v2 Announce Type: replace Abstract: Intravascular ultrasound (IVUS) lumen and external elastic membrane (EEM) segmentation is important for quantitative coronary plaque burden assessme

researcharxiv-cs-cv

Comparing Chunking and Embedding Strategies for Turkish RAG Systems

arXiv:2608.26192v1 Announce Type: cross Abstract: How documents are segmented into retrievable chunks and how those chunks are embedded strongly affect Retrieval-Augmented Generation (RAG) quality, ye

researcharxiv-cs-ai

Compositional Generalization via Structural Identification in a Category-Theoretic Framework

arXiv:2608.26465v1 Announce Type: new Abstract: Compositional generalization is usually evaluated through model accuracy. We instead ask which structural or lexical identifications make held-out COGS

DocTalkBN: A Novel Dataset of Expert Telemedicine Conversations in Bengali

arXiv:2608.27110v1 Announce Type: new Abstract: Reliable medical conversational AI requires authentic expert--patient interaction data, yet such datasets remain scarce, especially for low-resource lan

Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI

arXiv:2608.26442v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, ma

DRL: A Deterministic Relational Middleware Layer for Transaction-Safe Enterprise NL2SQL Under Schema-Graph Scaling

arXiv:2608.26172v1 Announce Type: cross Abstract: Deploying natural-language interfaces over enterprise OLTP catalogs fails at scale because semantic parsers collapse under schema-graph scaling, infla

FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation

arXiv:2608.24168v2 Announce Type: replace Abstract: A unified audio model must recognize and understand linguistic, paralinguistic, and environmental information while supporting speech synthesis and

Agents

Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training

arXiv:2608.26730v1 Announce Type: new Abstract: Large language models offer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Aut

agentsarxiv-cs-ai

LLMs Can Design Near-Optimal OR Algorithms

arXiv:2608.27296v1 Announce Type: new Abstract: We ask whether large language models (LLMs) can design effective algorithms for well-specified operations research (OR) problems. We study inventory con

Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation

arXiv:2608.26535v1 Announce Type: new Abstract: Audio-video generation is rapidly moving from prompt-driven synthesis toward multimodal conditioning, where text, images, audio, and video can jointly s

PragAlign: Evidence-Sensitive Reply Assistance Across Chinese and Japanese Appropriateness Judgments

arXiv:2608.26700v1 Announce Type: new Abstract: Reply assistance in multilingual settings requires linguistic competence and culturally situated judgments of appropriateness. We present PragAlign, whi

Prediction of Prediction (PoP): Inter-Layer Activation Fusion for Single-Pass Hallucination Detection in Large Language Models

arXiv:2608.27165v1 Announce Type: new Abstract: Autoregressive large language models (LLMs) routinely generate factually incorrect outputs with high decoding confidence, limiting their deployment in h

RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature

arXiv:2608.27394v1 Announce Type: new Abstract: Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly

Residual Deep Reinforcement Learning-Based Computed Torque Control for a Cable-Driven Lower-Limb Rehabilitation Robot under Disturbances and Parametric Uncertainties

arXiv:2608.26739v1 Announce Type: new Abstract: Accurate trajectory tracking in cable-driven lower-limb rehabilitation robots is challenging because model uncertainty, external disturbances, joint con

safetyarxiv-cs-ro

RuleWeaver: Benchmarking Rule-Centered Scenario Reasoning for Large Language Models

arXiv:2608.26832v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly applied to specialized domains, where effective use of domain expertise often requires reasoning over comp

Agents

SKILL.state: Scalable Long-Horizon Agent Skills

arXiv:2608.26263v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly act as autonomous agents executing complex, long-running procedural skills. Existing agent runtimes maintain e

agentsarxiv-cs-ai

Temporally-Grounded Language Generation: Towards Real-Time Vision-Language Models

arXiv:2505.11326v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) have shown remarkable progress in offline tasks such as image captioning and video question answering. However,

What the 'Spotless' Mind Remembers: How Knowledge Entanglement Shapes What Leaks After Unlearning in LLMs

arXiv:2510.25732v2 Announce Type: replace-cross Abstract: Unlearning in large language models (LLMs) is usually evaluated as whether an 'unlearned' fact can be recovered. We instead ask whether a fact

researcharxiv-cs-ai

A General-Purpose Molecular Foundation Model Transfers Across Diverse Olfactory Tasks

model-releasesarxiv-cs-lg

arXiv:2608.25893v1 Announce Type: new Abstract: Foundation models have transformed molecular property prediction, yet it remains unclear whether a molecular foundation model, fine-tuned on a single ca

A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

arXiv:2606.19660v2 Announce Type: replace-cross Abstract: Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applicatio

AutoVerifier: Residual-Guided Non-Parametric Optimization for Reference-Based Answer Verification

arXiv:2608.25637v1 Announce Type: new Abstract: Reference-based verifiers are important for evaluating reasoning models and providing accurate outcome rewards in reinforcement learning with verifiable

Can You Trust Frozen Hematology Foundation Models under Acquisition Shift?

arXiv:2608.25148v1 Announce Type: new Abstract: Frozen hematology foundation-model (FM) embeddings reach near-saturated in-domain white-blood-cell (WBC) accuracy, but clinical deployment demands relia

CEDAR: Controlled and Event-Driven Demand Forecasting via Residual Decomposition

arXiv:2608.25871v1 Announce Type: new Abstract: Forecasting in large-scale e-commerce marketplaces is increasingly required to support planning: merchants need to evaluate sales outcomes under future

safetyarxiv-cs-lg

CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact

arXiv:2608.25539v1 Announce Type: new Abstract: A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluate

Flower Hub: A Reproducible Benchmarking Platform for Federated Learning in Simulation and Deployment

model-releasesarxiv-cs-lg

arXiv:2608.25114v1 Announce Type: new Abstract: Federated learning (FL) has emerged as a key approach for training models across decentralized data, yet benchmarking in FL remains difficult to reprodu

Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures

arXiv:2608.25926v1 Announce Type: cross Abstract: Air traffic control procedures are executed through spoken exchanges between controllers and pilots. These interactions are essential to the safety of

safetyarxiv-cs-cl

From National Curricula to Cultural Awareness: Constructing Open-Ended Culture-Specific Question Answering Dataset

arXiv:2601.04632v2 Announce Type: replace Abstract: Large language models (LLMs) achieve strong performance on many tasks, but their progress remains uneven across languages and cultures, often reflec

safetyarxiv-cs-cl

GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Prediction

arXiv:2601.13207v2 Announce Type: replace Abstract: Geo-localization aims to infer the geographic location where an image was captured using observable visual evidence. Traditional methods achieve imp

Key Point Analysis Needs Structure Recovery: Task Definition, Dataset Diagnosis, and a Structure-Aware Benchmark

arXiv:2608.25854v1 Announce Type: new Abstract: Key Point Analysis (KPA) aims to identify a concise set of key points that summarize a collection of arguments together with their prevalence. We argue

Applications

Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy

arXiv:2608.25810v1 Announce Type: new Abstract: Foundation models are increasingly deployed for medical image analysis. However, under the inter-institutional distribution shift typical of deployment,

applicationsarxiv-cs-cv

Local Ai

LLMTrace: A Corpus for Classification and Fine-Grained Localization of AI-Written Text

arXiv:2509.21269v2 Announce Type: replace Abstract: The widespread use of human-like text from Large Language Models (LLMs) necessitates the development of robust detection systems. However, progress

local-aiarxiv-cs-cl

LUTSeg: A Longitudinal Multi-Expert Dataset for Ulcer Tissue Segmentation

arXiv:2608.25866v1 Announce Type: new Abstract: Quantifying wound tissue composition is essential for monitoring chronic ulcer progression and guiding treatment decisions. However, pixel-level annotat

OmniPhys: A Unified Multimodal Benchmark for Physics Understanding and Generation from Chinese Educational Corpora

arXiv:2608.25398v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong abilities in solving diverse visual and textual reasoning tasks. However, their develo

ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architectures

arXiv:2510.04938v2 Announce Type: replace-cross Abstract: Neural architecture search (NAS) automates the design process of high-performing architectures, but remains bottlenecked by expensive performa

PhaseShift: Topology-Aware Data Harmonization and Model Consolidation Across Signalized Intersections

arXiv:2608.25275v1 Announce Type: cross Abstract: Learned traffic-behavior models are commonly trained separately for each intersection, creating model portfolios that cannot share evidence across sit

researcharxiv-cs-ro

PointRL: Learning Point-Level Vision-Language Grounding from Verifiable Annotation Evidence

arXiv:2608.25299v1 Announce Type: new Abstract: Vision-language models (VLMs) increasingly rely on point coordinates as a compact and executable interface for visual grounding in GUI interaction, robo

researcharxiv-cs-cv

SciMIF: Understanding Multimodal Instruction Following in Scientific Domains

model-releasesarxiv-cs-lg

arXiv:2608.25973v1 Announce Type: cross Abstract: Understanding instruction-following capabilities in scientific domains is essential for effectively leveraging Multimodal Large Language Models (MLLMs