Search: “evaluation”

SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs

arXiv:2608.07862v1 Announce Type: new Abstract: Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguist

11 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

TCS-BENCH: Benchmarking State-of-the-Art Generative AI Theoretical Computer Science Research Ability

DGX agent

arXiv:2608.09538v1 Announce Type: cross Abstract: We introduce TCS-Bench, a benchmark for evaluating Large Language Models (LLMs) on research-level Theoretical Computer Science (TCS) proof generation.

model-releasesarxiv-cs-ai

11 Aug 2026

Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution

arXiv:2608.06811v1 Announce Type: cross Abstract: Resolving a real software issue with a large language model (LLM) agent is a long repair episode, often tens to hundreds of steps spanning exploration

Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

arXiv:2608.06955v1 Announce Type: new Abstract: Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs sys

GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

arXiv:2608.06769v1 Announce Type: new Abstract: Recent Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse vision-language tasks, creating an urgent need for more

How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality

arXiv:2604.06756v2 Announce Type: replace Abstract: Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible

researcharxiv-cs-cl

Let's Unlearn Stereotypes Before Decision-Making: Assessing the Impact of Intrinsic Bias Mitigation on Downstream Fairness in LLMs

arXiv:2509.16462v2 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly used in high-stakes decision-making systems, where biased predictions can reinforce social and economi

safetyarxiv-cs-cl

Multi-Perspective Triad Interaction Graph Neural Network for Cognitive Distortion Detection

arXiv:2608.06785v1 Announce Type: new Abstract: Cognitive distortion detection is a key task in computational mental health, yet existing approaches often overlook the psychological structure of disto

safetyarxiv-cs-cl

UniREditBench: A Unified Reasoning-based Image Editing Benchmark

arXiv:2511.01295v3 Announce Type: replace Abstract: Recent advances in multi-modal generative models have driven substantial improvements in image editing. However, current generative models still str

WebRider: Persona-Conditioned Intent Controllers for Live-Web Assistance

arXiv:2608.06704v1 Announce Type: new Abstract: Delegating a web task involves more than asking a question; it requires transferring a policy: what to verify, how to handle uncertainty, which preferen

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

arXiv:2608.07051v1 Announce Type: new Abstract: Generic parameter-efficient fine-tuning (PEFT) methods transferred from language models can fail silently on real-time detectors, whose heterogeneous op

Clinician input steers AI toward accurate and harmful recommendations

model-releasesarxiv-cs-lg

arXiv:2603.14158v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are entering clinical workflows, yet evaluations rarely assess how clinician reasoning shapes model behavior duri

EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?

arXiv:2608.06022v1 Announce Type: new Abstract: Epitopes determine where antibodies bind antigens and shape downstream therapeutic properties such as functional blockade and escape resistance, making

From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs

arXiv:2608.05560v1 Announce Type: cross Abstract: Timely anticipation of physical hazards is essential for real-world safety, yet existing MLLM evaluations focus on harmful content or general risks, l

Hypothesis Testing with Conditional Queries: Learnability and the Value of Interaction

arXiv:2608.06262v1 Announce Type: new Abstract: Model evaluations may fix all tests before observing any responses or select later tests using earlier responses. We study this choice in a conditional-

safetyarxiv-cs-lg

Integrating Human Linguistic Insights into AI: Theory-Driven Representation for Multilingual Text-to-Speech

arXiv:2204.07228v2 Announce Type: replace Abstract: This paper explores the integration of human linguistic insights into multilingual text-to-speech (TTS) systems by evaluating the Featurally Undersp

researcharxiv-cs-cl

LUNAR: Benchmarking Personalized Large Language Models on UNiversal User BehAvioR Logs

arXiv:2608.05246v1 Announce Type: new Abstract: Existing personalized LLM benchmarks primarily rely on textual personas or isolated behavioral signals, providing limited evaluation of cross-domain beh

MAC 2026: Advancing Micro-Action Analysis Towards Fine-Grained Understanding

arXiv:2607.16284v2 Announce Type: replace Abstract: Micro-Actions (MAs) are subtle and spontaneous human behaviors that provide important non-verbal cues in social interaction and affective communicat

MicroEvo: Knowledge-Guided LLM Sampling for Efficient Microarchitecture Design Space Exploration

arXiv:2608.06183v1 Announce Type: new Abstract: Microarchitecture design space exploration suffers from expansive search spaces and expensive PPA evaluation, leaving only a small simulation budget for

safetyarxiv-cs-ai

NavTrust: Benchmarking Trustworthiness for Embodied Navigation

arXiv:2603.19229v2 Announce Type: replace-cross Abstract: There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language i

Physics-Based Molecular Fingerprints from Spectral Graph Theory Provide Efficient Geometry-Aware Measures of Chemical Similarity

arXiv:2608.05336v1 Announce Type: cross Abstract: Molecular representations are essential for the evaluation of molecular similarity and the development of structure-property relationships. Despite th

researcharxiv-cs-lg

SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution

arXiv:2608.05573v1 Announce Type: new Abstract: LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to veri

Tutorials

SR-JEPA: Learning Predictive Latent State in 3D Scenes

arXiv:2608.05774v1 Announce Type: new Abstract: Joint-embedding predictive architectures learn by predicting latent representations of missing observations, yet many masked JEPAs are evaluated primari

tutorialsarxiv-cs-cv

TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding

arXiv:2608.05699v1 Announce Type: new Abstract: Humans understand anomalous events through a coherent perceptual process in which they identify the focal instance, follow its behavior as the event unf

XEWorld: Can Action-Conditioned World Models Generalize to Unseen Robot Embodiments?

arXiv:2608.05799v1 Announce Type: cross Abstract: Action-conditioned world models are promising learned simulators for robotic manipulation, yet evaluating them exclusively on training robots fails to

safetyarxiv-cs-cv

A Multi-Cohort Validation of Censoring-Aware Conformal Lower Predictive Bounds for Pathology Survival Models

arXiv:2608.04025v1 Announce Type: cross Abstract: Whole-slide survival models commonly provide risk rankings without calibrated statements about individual event times. We evaluate fixed-cutoff drcosa

researcharxiv-cs-lg

Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports

arXiv:2608.04682v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) are increasingly adopted in software engineering (SWE) scenarios, capable of fixing a specific b

Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings

arXiv:2608.04735v1 Announce Type: new Abstract: Chain-of-thought (CoT) monitoring is increasingly treated as an important safety layer for frontier reasoning models. Most monitorability evaluations st

DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots

arXiv:2608.05004v1 Announce Type: new Abstract: Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs), including 'delusi

Diagnosing Tool-Selection Reasoning in LLM Agents with Canary Tools

arXiv:2608.04719v1 Announce Type: new Abstract: Agent evaluations tell us that a model picked the wrong tool, but rarely why. We introduce canary tools: diagnostic probe tools planted in an agent's Mo

From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models

model-releasesarxiv-cs-lg

arXiv:2608.04200v1 Announce Type: cross Abstract: Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically

InvFlowFD: Reference-Free and Background-Set-Free Perceptual Music Quality Metric with Flow Matching Inversion

arXiv:2608.04142v1 Announce Type: cross Abstract: Existing reference-free methods for evaluating music perceptual quality alleviate the need for paired noisy-clean data, but they still rely on a backg

researcharxiv-cs-ai

Item Response Theory for AI Safety

arXiv:2608.05086v1 Announce Type: new Abstract: Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to tr

Learning to Resolve Neutron Resonances with Fully Convolutional Neural Networks

arXiv:2608.04027v1 Announce Type: new Abstract: This work investigates the feasibility of augmenting traditional R-Matrix codes with a robust machine learning framework for automatically detecting neu

safetyarxiv-cs-lg

MGSB: Manifold Gated Signature Branch Pressure-Domain Baseline Architecture for Two-Phase Pipeline Flows Under Distributional Shift

arXiv:2608.04805v1 Announce Type: new Abstract: Leak detection models for multiphase pipelines often degrade when deployed under flow regimes that differ from training. Existing evaluations typically

safetyarxiv-cs-lg

Mind the Cap: Output-Budget Regimes Change the Measured Multilingual Reasoning Gap

arXiv:2608.04160v1 Announce Type: new Abstract: Multilingual evaluations report accuracy at a single output-token cap, but languages need different numbers of tokens to express the same content, so th

Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs

arXiv:2608.04501v1 Announce Type: new Abstract: Video surveillance in public safety, healthcare, and smart environments has made continuous human monitoring routine, raising real risks to personal ide

Robust and Personalized Federated Learning for Aircraft-Engine Prognostics under Benign and Adversarial Client Heterogeneity

arXiv:2608.04045v1 Announce Type: cross Abstract: Federated learning (FL) enables aircraft fleet operators to jointly train remaining-useful-life (RUL) models from engine sensor telemetry without shar

SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models

arXiv:2608.04244v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) make grounded predictions in real-world scenes by combining visual and textual cues, yet existing benchmarks

SSC: A Verifiable Structured Representation for Bimanual Manipulation Labelling

arXiv:2608.04425v1 Announce Type: new Abstract: Subtask labels decompose a long-horizon manipulation demonstration into shorter semantic segments for policy training and evaluation. Natural language d

safetyarxiv-cs-ro

The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answering

arXiv:2608.04589v1 Announce Type: cross Abstract: EgoCross is a cross-domain egocentric video question answering benchmark designed to evaluate whether multimodal large language models can generalize

Trident : How to Break Deep Reinforcement Learning Cyber Defenses (Agentic)

arXiv:2608.04317v1 Announce Type: cross Abstract: Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost

Agents

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

arXiv:2608.00155v1 Announce Type: cross Abstract: Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominan

agentsarxiv-cs-lg

Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson

arXiv:2608.03938v1 Announce Type: new Abstract: Bimanual manipulation policies trained with imitation learning are typically evaluated on workstation or datacenter-class GPUs, leaving the cost of depl

safetyarxiv-cs-ro

CARE-Bench: Benchmarking Patient-Facing LLM Triage

arXiv:2608.03731v1 Announce Type: new Abstract: Patient-facing medical LLMs and agents increasingly answer symptom questions before clinician contact, where the key safety question is what action the

Confident but Unreliable: A Behavioral Safety Audit of Vision-Language Models on Brain MRI

arXiv:2608.02790v1 Announce Type: new Abstract: Vision-language models (VLMs), including medical specialists, are increasingly proposed for medical imaging, yet their stated confidence is rarely evalu

Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili

arXiv:2608.03532v1 Announce Type: new Abstract: Large language models are increasingly deployed in multilingual contexts, yet safety alignment and bias evaluation remain overwhelmingly English-centric

Geo-Embed: Towards Unified Multimodal Embeddings for Urban Understanding

arXiv:2608.03826v1 Announce Type: new Abstract: Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations,