Search: “evaluation”

Investigating the Ability of Large Language Models to Analyze Recipes for Diabetes

arXiv:2609.03967v1 Announce Type: cross Abstract: Several studies have evaluated the ability of Large Language Models (LLMs) for meal planning, yielding positive outcomes. These models can process nat

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Measuring Harmfulness of Computer-Using Agents

DGX agent

arXiv:2508.00935v3 Announce Type: replace-cross Abstract: Computer-using agents (CUAs), which can autonomously control computers to perform multi-step actions, might pose significant safety risks if m

model-releasesarxiv-cs-ai

4 Sept 2026

MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT

arXiv:2609.03690v1 Announce Type: new Abstract: The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advanc

Preprocessing Failure and Adversarial Detection in Depthwise-Separable Edge Vision Systems

arXiv:2609.03453v1 Announce Type: new Abstract: Preprocessing-based defenses are the standard first-line response to adversarial attacks on edge vision systems, requiring no retraining, no architectur

researcharxiv-cs-cv

Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting

arXiv:2609.03923v1 Announce Type: new Abstract: In online meeting delegation, LLM agents fail to recognize when to speak. With no structured way to track stances, coverage, and floor, they miss the mo

agentsarxiv-cs-ai

To What Extent Do Large Language Models Understand Bangla Idioms?

arXiv:2609.03410v1 Announce Type: new Abstract: Idiomatic expressions are an integral part of natural language, reflecting cultural nuances and posing unique challenges for computational models, parti

A Physics-Consistent Benchmark for Contact-Rich Human-Robot Interaction in Assistive Care

model-releasesarxiv-cs-ro

arXiv:2609.02402v1 Announce Type: new Abstract: Conventional task-level evaluation asks whether a robot policy completes a specified action, but can miss failures that emerge only during physical huma

Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models

arXiv:2609.02502v1 Announce Type: cross Abstract: Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce vis

CivBench: A Long-Horizon Benchmark for Tool-Mediated Agents in Civilization VI

arXiv:2609.02459v1 Announce Type: new Abstract: We present CivBench, an open-source benchmark for evaluating language model agents in long-horizon, tool-mediated environments through the Model Context

Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment

arXiv:2609.02573v1 Announce Type: new Abstract: Current evaluations and training of multimodal models predominantly focus on multi-image tasks, largely overlooking interleaved text-image scenarios. In

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

arXiv:2609.01691v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework fo

HarmReduction: Benchmarking LLMs in Harm Reduction Information Provision to Support People Who Use Drugs

arXiv:2507.21815v2 Announce Type: replace Abstract: Millions of individuals' well-being are challenged by the harms of substance use. Harm reduction as a public health strategy provides non-judgementa

PragAlign: Feedback-Guided Pragmatic Alignment for Controlled Synthetic Dialogue Generation

arXiv:2609.02480v1 Announce Type: new Abstract: Synthetic dialogue generation can support research in privacy-restricted service settings, but generated conversations must preserve communicative inten

safetyarxiv-cs-cl

UTP-Bench: Uncertainty-aware Travel Planning Benchmark

arXiv:2609.02421v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planni

A Benchmark for Vehicle Attribute Classification in Cross-Domain Surveillance Scenarios

arXiv:2609.01584v1 Announce Type: new Abstract: Vehicle attribute analysis is a key component of Intelligent Transportation Systems (ITS), supporting applications such as vehicle identification, traff

Agentic Empirical Asset Pricing: Methodological Foundations

arXiv:2609.00731v1 Announce Type: new Abstract: Recent advances in LLM agents enable a new paradigm for asset pricing, which we call Agentic Empirical Asset Pricing (AEAP): systems that autonomously c

agentsarxiv-cs-ai

Are Near-Tied LLM Rankings Robust to Family-DIF-Guided Benchmark Recomposition?

arXiv:2609.00482v1 Announce Type: new Abstract: Small leaderboard gaps are often interpreted as evidence that one language model is better than another, but their sign may depend on which benchmark it

Conditional Flow Matching for ML-Based Inverse Design Problems

arXiv:2609.00863v1 Announce Type: new Abstract: Engineering inverse design is often limited by the high computational cost of iterative solvers for optimization problems constrained by partial differe

researcharxiv-cs-lg

CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs

model-releasesarxiv-cs-lg

arXiv:2609.01161v1 Announce Type: new Abstract: Large language models can reproduce memorized text verbatim, yet copyright defenses are usually evaluated under incompatible protocols. We introduce Cop

GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments

arXiv:2609.00048v1 Announce Type: cross Abstract: GUI world models are increasingly evaluated as one-step next-screen predictors, yet their intended use is often as multi-step environments for GUI age

Measuring the Behavioral Fidelity of Long-Horizon Human Activity Simulations

arXiv:2609.01257v1 Announce Type: new Abstract: As LLM-based human simulators are increasingly used for policy, evaluation, and training, they must faithfully reproduce real behavioral patterns. While

safetyarxiv-cs-ai

NSIDDx: A Design Framework for Neuro-Symbolic, Practitioner-First Differential Diagnosis in Low-Resource Settings

arXiv:2609.00256v1 Announce Type: new Abstract: LLM-based diagnostic systems achieve high semantic accuracy on benchmarks, but open-ended evaluation on clinically uncommon presentations reveals a syst

agentsarxiv-cs-cl

Optimizing Byzantine Node Placement in Decentralized Federated Learning

arXiv:2609.01495v1 Announce Type: cross Abstract: Security evaluations of decentralized federated learning (DFL) typically focus on how Byzantine participants behave, while largely overlooking which p

ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues

arXiv:2606.18237v2 Announce Type: replace-cross Abstract: Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to eva

Scale-based Approach for Active Wildfire Segmentation on Satellite Imagery

arXiv:2609.01392v1 Announce Type: new Abstract: Active wildfire mapping from satellite imagery is challenging due to the sparse and highly imbalanced nature of fire pixels, especially in early-stage o

researcharxiv-cs-cv

Local Ai

The Answer Is Not the Argument

arXiv:2609.00264v1 Announce Type: new Abstract: Chain-of-thought monitoring is proposed for AI oversight, yet evaluations often provide monitors with a trusted reference answer. We ask whether answer

local-aiarxiv-cs-ai

Toward Workflow-Aware Benchmarking for Healthcare NLP Agents

arXiv:2609.00296v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly proposed for healthcare tasks such as clinical documentation, evidence retrieval, patient messaging,

agentsarxiv-cs-cl

Visual Framing for News Stance Detection via Image Generation

arXiv:2609.00685v1 Announce Type: cross Abstract: Article-level news stance detection aims to identify the perspective of news articles toward social issues. Despite advances in stance detection and i

AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds

arXiv:2608.29397v1 Announce Type: new Abstract: Tool-use benchmarks generally evaluate whether an agent completes a workflow using appropriate tools and valid arguments. However, feasibility alone is

Arkios: An Open Bilingual English-Nepali Language Model Trained From Scratch, with a Devanagari-Aware Tokenizer

arXiv:2608.30092v1 Announce Type: cross Abstract: We present Arkios, a 1.04B-parameter dense transformer pretrained from scratch on 150B tokens of bilingual English-Nepali text, using a custom single-

Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational Impact in LLM Tutors

arXiv:2606.16206v2 Announce Type: replace Abstract: Large language models are increasingly proposed as educational tutors, yet stronger task-solving ability does not necessarily imply stronger learnin

DERELAB: Probing Defeasible Reasoning and Confirmation Bias in LLMs with a Generative Benchmark

arXiv:2608.30413v1 Announce Type: new Abstract: Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of ne

Error Detection for PET/CT Radiology Reports: Domain-Specific vs Large Language Models

arXiv:2608.30021v1 Announce Type: cross Abstract: Errors in radiology reports can adversely affect patient treatment, yet automated report quality assurance remains challenging because errors are ofte

From Analytics to Tumor Boards: An Evidence-Linked Multi-Agent Workflow for Oncology Feature Extraction

arXiv:2608.28974v1 Announce Type: new Abstract: Clinically relevant oncology information is distributed across heterogeneous, longitudinal documentation, creating substantial abstraction burden and re

agentsarxiv-cs-ai

Generating Clinical Vignettes that Preserve Cognitive Formulations

arXiv:2608.29995v1 Announce Type: cross Abstract: Large language models can generate fluent clinical case vignettes, but fluency alone does not ensure fidelity to a specifiable clinical structure. We

GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space

arXiv:2608.30188v1 Announce Type: new Abstract: Large Language Models (LLMs) show great potential as clinical agents, yet existing benchmarks reduce clinical workflows to static predictions or unconst

Learning Compositional Spatio-Temporal Video Grounding with Synthetic Curriculum

arXiv:2608.30584v1 Announce Type: new Abstract: Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on sim

Motus2: A Self-Evolving General World Model for Dexterous Manipulation

arXiv:2608.30237v1 Announce Type: cross Abstract: General embodied agents should perceive, predict, act, evaluate, and improve within a unified system. World models have shown great promise in buildin

safetyarxiv-cs-ai

N_0-Foundation: Towards the Age of Tactile Intelligence

arXiv:2608.29601v1 Announce Type: cross Abstract: We present N_0-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal dat

Redesigning and Auditing Deep Research Writing for Faithful Reports

arXiv:2608.28643v1 Announce Type: cross Abstract: Rubric-based evaluations of deep-research (DR) systems often obscure fine-grained factual failures in generated reports. We introduce CLAIMPROBE, a cl

SurgSkill-Bench: A Benchmark for Multimodal Surgical Skill Assessment

arXiv:2608.30872v1 Announce Type: new Abstract: Objective assessment of surgical technical skill is important for surgical training and structured feedback, but current workflows remain dependent on l

Applications

Uncertainty-Aware Trajectory Forecasting from Imperfect Tracking

arXiv:2608.30899v1 Announce Type: new Abstract: Most trajectory forecasting models are trained on clean annotated histories, and are often evaluated under the same idealized assumption, although pract

applicationsarxiv-cs-cv

VIBE: Video Instruction-aligned Background music gEneration

arXiv:2608.30125v1 Announce Type: cross Abstract: Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstruction

safetyarxiv-cs-ai

Local Ai

XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals

arXiv:2608.29948v1 Announce Type: new Abstract: Evaluating data-text alignment remains challenging: existing metrics often provide limited explanations for the scores, while prompt-based LLM-as-Judge

local-aiarxiv-cs-cl

Denoising-Aware Temporal Point Cloud Completion for 3D Crop Architecture Recovery and Phenotypic Trait Extraction

arXiv:2608.28343v1 Announce Type: new Abstract: High-throughput phenotyping depends on accurate 3D reconstruction of plants across growth stages, yet the development and evaluation of temporal complet

31 Aug 2026

Explainable Diabetic Retinopathy Classification Using Vision Foundation Models

arXiv:2608.28207v1 Announce Type: new Abstract: Diabetic retinopathy (DR) is a major cause of preventable blindness, creating a need for accurate and trustworthy automated screening. This study invest

31 Aug 2026

A Multi-Framework Comparison of Outline Stages in Long-Form Generation with LLMs

arXiv:2608.26177v1 Announce Type: cross Abstract: Long-form generation exposes fundamental limitations of large language models. Even 70B-parameter models exhibit length collapse at 16k-token outputs,

28 Aug 2026

AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition

arXiv:2608.26434v1 Announce Type: new Abstract: Code-switching is pervasive in bilingual African conversation, yet most ASR systems assume monolingual input and are evaluated on curated monolingual be