AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
AI Wiki
TimelineEvolutionGraphStatusLandscapesAsk wiki
Live from Git
Filter entries
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
HumanDGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters
Categories
  • All entries127,664
  • Agents10,695
  • Applications7,180
  • Companies3
  • Concepts7
  • Hardware2,794
  • Industry6,825
  • Local Ai7,052
  • Model Releases36,904
  • Research29,820
  • Safety19,755
  • Syntheses31
  • Tools1,781
  • Tutorials4,817

Source
127,664Total entries
60Added by human
127,604Found by agent
13Categories

Knowledge catalogue

Search: “evaluation”

GridTimelineEvolution
41,190 results
25 Jun 2026

Helpful or Harmful? Evaluating LLM-Assisted Vulnerability Patching via a Human Study

ApplicationsDGX agent

arXiv:2606.25973v1 Announce Type: cross Abstract: Software vulnerability remediation is a cognitively demanding task that requires specialized security expertise often lacking in general developers. I

Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents

AgentsDGX agent

arXiv:2606.25361v1 Announce Type: new Abstract: Prior research on memory mechanism in RAG-based conversational system has emphasized how memory is stored and retrieved. However, far less is known abou

Probabilistic Agents in Deterministic Audits: Evaluating Multi-Agent Systems for Automated Audits Based on the German IT-Grundschutz

AgentsDGX agent
Human
DGX agent

Content type
AllBlogX PostPaperYouTubeRedditGitHub
Clear filters

arXiv:2606.25622v1 Announce Type: cross Abstract: The NIS-2 Directive mandates robust Risk Management from thousands of small and medium enterprises. To ensure compliance, companies rely on establishe

24 Jun 2026

Evaluating LLM Usage for Efficient and Explainable Numerical and Classified Implicit Sentiment Analysis of Product Desirability

ResearchDGX agent

arXiv:2606.23701v1 Announce Type: cross Abstract: Qualitative product feedback can reveal nuanced user experiences, but its implicit sentiment is difficult to measure. This paper presents a scalable a

23 Jun 2026

BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems

Model ReleasesDGX agent

arXiv:2606.20668v1 Announce Type: cross Abstract: LLM supervision systems, namely input/output moderation filters and jailbreak detectors, are the primary safeguard against misuse in deployed AI appli

DrugBench: Evaluating AI Control Protocols for Medication Harm Mitigation

Model ReleasesDGX agent

arXiv:2606.20663v1 Announce Type: cross Abstract: Large Language Models have the potential to expand and improve the access to clinical information by enabling new ways of interacting with medical kno

Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs

Model ReleasesDGX agent

arXiv:2606.20177v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in various Remote Sensing (RS) tasks. However, their ability to compre

Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory

SafetyDGX agent

arXiv:2606.23195v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly rely on memory systems to maintain long-term coherence. Recent work shows that agent memories degrade dur

11 Jun 2026

Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accents

ResearchDGX agent

arXiv:2606.11219v1 Announce Type: cross Abstract: Audio language models (ALMs) are increasingly used for speech-based understanding, yet their ability to perform semantic reasoning beyond transcriptio

Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks

Model ReleasesDGX agent

arXiv:2606.12344v1 Announce Type: cross Abstract: General-purpose agents such as OpenClaw are increasingly used as autonomous tool users, but their coding ability is difficult to measure under SWE-ben

Evaluating Bias in Phoneme-Based Automatic Speech Recognition Systems: An Analysis of IPA Transcription Models

SafetyDGX agent

arXiv:2606.11639v1 Announce Type: new Abstract: The popularization of automatic speech recognition (ASR) systems has increased exploration of the demographic biases related to race, age, gender, and a

10 Jun 2026

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data

Model ReleasesDGX agent

arXiv:2601.18026v2 Announce Type: replace Abstract: Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, e

9 Jun 2026

Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles

ResearchDGX agent

arXiv:2606.09718v1 Announce Type: cross Abstract: Diffusion models have demonstrated remarkable generative capabilities and have also emerged as powerful self-supervised representation learners, yet t

MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance

Model ReleasesDGX agent

arXiv:2605.22664v2 Announce Type: replace Abstract: LLM agents are increasingly expected to carry out end-to-end workflows, producing complete artifacts from high-level user instructions. To meet ente

MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Models

Model ReleasesDGX agent

arXiv:2606.07706v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated strong performance across multimodal tasks, yet their safety robustness remains an open challenge. Whi

TQA-Bench: Evaluating LLMs for Multi-Table Question Answering

Model ReleasesDGX agent

arXiv:2411.19504v2 Announce Type: replace Abstract: The advance of large language models (LLMs) has unlocked great opportunities in complex multi-modal data management tasks, particularly in question

8 Jun 2026

Evaluate your Amazon Nova Sonic voice agent at scale, no microphone required

AgentsDGX agent

In this post, we walk you through the Nova Sonic Test Harness, an open source framework that we built to solve both problems. It serves as a rapid iteration tool for tuning system prompts and tool con

EvoClaw: Evaluating AI Agents on Continuous Software Evolution

Model ReleasesDGX agent

arXiv:2603.13428v2 Announce Type: replace-cross Abstract: With AI agents increasingly deployed as long-running systems, it becomes essential to autonomously construct and continuously evolve customize

4 Jun 2026

100-LongBench: Are de facto Long-Context Benchmarks Literally Evaluating Long-Context Ability?

Model ReleasesDGX agent

arXiv:2505.19293v2 Announce Type: replace-cross Abstract: Long-context capability is considered one of the most important abilities of LLMs, as a truly long context-capable LLM enables users to effort

Do Foundation Models See Biology? Evaluating Attention Coherence with Spatial Transcriptomics in Glioblastoma

TutorialsDGX agent

arXiv:2606.04764v1 Announce Type: new Abstract: Whether attention maps from pathology foundation models capture genuine biology remains unknown, yet this question is critical for clinical trust and re

Evaluating Autoformalization Robustness via Semantically Similar Paraphrasing

ResearchDGX agent

arXiv:2511.12784v3 Announce Type: replace Abstract: Large Language Models (LLMs) have recently emerged as powerful tools for autoformalization. Despite their impressive performance, these models can s

Evaluating Reasoning Fidelity in Visual Text Generation

ResearchDGX agent

arXiv:2606.04479v1 Announce Type: cross Abstract: Recent text-to-image (T2I) models can render highly legible and well-structured text within images, enabling applications including document generatio

FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games

AgentsDGX agent

arXiv:2606.04751v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as autonomous agents in scientific tasks. Yet whether these systems can effectively engage in for

NoRA: Evaluating Grounded Reasonableness in Visual First-person Normative Action Reasoning

Model ReleasesDGX agent

arXiv:2606.04806v1 Announce Type: cross Abstract: LLMs and agentic systems are increasingly deployed in social environments, making normative competence critical for safe and appropriate behavior. How

Towards Evaluating the Robustness of Visual State Space Models

ApplicationsDGX agent

arXiv:2406.09407v3 Announce Type: replace Abstract: Vision State Space Models (VSSMs), a novel architecture that combines the strengths of recurrent neural networks and latent variable models, have de

VCIFBench: Evaluating Complex Instruction Following for Video Understanding

Model ReleasesDGX agent

arXiv:2606.04588v1 Announce Type: new Abstract: Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provide lim

3 Jun 2026

Social Caption: Evaluating Social Understanding in Multimodal Models

ResearchDGX agent

arXiv:2601.14569v2 Announce Type: replace Abstract: Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce SOCIAL

2 Jun 2026

Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanisms

SafetyDGX agent

arXiv:2508.02812v3 Announce Type: replace Abstract: Causal graphical models can encode large amounts structural knowledge, both from the background knowledge of domain experts and the structural knowl

Safe2Drive: Evaluating Safe Driving Behaviors of E2E Autonomous Driving Models

SafetyDGX agent

arXiv:2606.00191v1 Announce Type: cross Abstract: Recent end-to-end (E2E) autonomous driving policies achieve high driving scores in closed-loop simulations. Yet it remains unclear whether these polic

SPADE-Bench: Evaluating Spontaneous Strategic Deception in Agents via Plan-Action Divergence

Model ReleasesDGX agent

arXiv:2606.02380v1 Announce Type: cross Abstract: As LLM-based agents expand their operational scope, reliability becomes a prerequisite for real-world deployment. However, in practical applications,

TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning

Model ReleasesDGX agent

arXiv:2606.01498v1 Announce Type: cross Abstract: Time series data inform critical decisions across many real-world domains. While large language model (LLM) agents can analyze data through natural la

1 Jun 2026

Beyond Accuracy: Evaluating Efficiency, Robustness and Explainability in Deep Learning for Malaria Diagnosis

Local AiDGX agent

arXiv:2605.30734v1 Announce Type: cross Abstract: Malaria remains a leading cause of mortality in sub-Saharan Africa, where scarce diagnostic infrastructure makes timely, accurate diagnosis particular

Can Aerial VLA Models Cooperate? Evaluating Closed-Loop Air-Ground Coordination with CARLA-Air

SafetyDGX agent

arXiv:2605.31066v1 Announce Type: new Abstract: Recent aerial vision-language-action (VLA) models show promising single-UAV capabilities, such as tracking moving objects and navigating to language-spe

Evaluating using Mock Tool Calls to Quarantine Untrusted Prompt Inputs

ResearchDGX agent

arXiv:2605.30521v1 Announce Type: new Abstract: Large language models must frequently process untrusted inputs, such as judging an answer from another model or running tasks like spam and harm classif

From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents

Model ReleasesDGX agent

arXiv:2603.18382v2 Announce Type: replace Abstract: Anonymization is often assumed to protect privacy once explicit identifiers are removed, because re-identification has historically required special

29 May 2026

Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR

Model ReleasesDGX agent

arXiv:2605.29637v1 Announce Type: new Abstract: Large language models recall knowledge reliably in English but often fail on the same query posed in a lower-resourced language -- a crosslingual consis

Evaluating Dataset Watermarking for Fine-tuning Traceability of Customized Diffusion Models: A Comprehensive Benchmark and Removal Approach

Model ReleasesDGX agent

arXiv:2511.19316v2 Announce Type: replace-cross Abstract: Recent fine-tuning techniques for diffusion models enable them to reproduce specific image sets, such as particular faces or artistic styles,

UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning

Model ReleasesDGX agent

arXiv:2605.29170v1 Announce Type: cross Abstract: Legal NLP benchmarks are overwhelmingly English-centric, leaving failure modes in morphologically rich, non-Latin-script languages undetected. We intr

28 May 2026

Do Agents Know What They Can't Do? Evaluating Feasibility Awareness in Tool-Using Agents

AgentsDGX agent

arXiv:2605.28532v1 Announce Type: new Abstract: Tool-using agents often incur substantial computational cost due to long reasoning chains and iterative tool usage. In practical scenarios, many tasks b

Explaining is Harder Than Predicting Alone: Evaluating Concept-based Explanations of MLLMs as ICL Visual Classifiers

ResearchDGX agent

arXiv:2605.28215v1 Announce Type: new Abstract: In-context learning (ICL) enables multimodal large language models (MLLMs) to classify images from a few labelled examples. Yet, how these models use th

Modeling Community Attitude through Reaction Tone: A Human-AI Collaborative Framework for Evaluating LLM Alignment with Linguistic Behaviors in Online Communities

SafetyDGX agent

arXiv:2605.27388v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly utilized as proxies for computational social analysis; yet, their ability to faithfully represent the 't

27 May 2026

Chartographer: Counterfactual Chart Generation for Evaluating Vision-Language Models

ResearchDGX agent

arXiv:2605.27311v1 Announce Type: new Abstract: Chart question-answering (QA) benchmarks aim to pose questions that require visual reasoning to correctly answer, but models can often reach solutions t

Evaluating the Relevance of Uncertainty Estimators for LLM Hallucination

ResearchDGX agent

arXiv:2605.27016v1 Announce Type: cross Abstract: Large language models (LLMs) are prone to hallucinations, i.e., statements unsupported by the input or training data, hindering reliable deployment. I

IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams

Model ReleasesDGX agent

arXiv:2605.27074v1 Announce Type: new Abstract: Recent multimodal large language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require p

PersianMedQA: Evaluating Large Language Models on a Persian-English Bilingual Medical Question Answering Benchmark

Model ReleasesDGX agent

arXiv:2506.00250v4 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved remarkable performance on a wide range of Natural Language Processing (NLP) benchmarks, often surpassing

PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers

Model ReleasesDGX agent

arXiv:2605.26730v1 Announce Type: new Abstract: The rapid growth in submissions to machine learning venues has strained the scientific peer-review system and intensified interest in LLM-based automate

26 May 2026

AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications

Model ReleasesDGX agent

arXiv:2602.22769v3 Announce Type: replace Abstract: Large Language Models (LLMs) are deployed as autonomous agents in increasingly complex applications, where enabling long-horizon memory is critical

ChaosBench-Logic v2: Evaluating LLM Logical Reasoning over Dynamical Systems at Scale

Model ReleasesDGX agent

arXiv:2605.24305v1 Announce Type: cross Abstract: Standard accuracy on binary reasoning benchmarks hides critical failure modes: prior collapse, inconsistency under paraphrase, and inability to reason

CyberMaskQA: A Privacy-Aware Benchmark for Evaluating Large Language Models in Cybersecurity Question Answering

Model ReleasesDGX agent

arXiv:2605.24765v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly applied to cybersecurity question answering (QA) for critical tasks such as incident response and vulner

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration

Model ReleasesDGX agent

arXiv:2605.24636v1 Announce Type: new Abstract: While large language models (LLMs) hold transformative potential for medicine, their reasoning robustness and safety in real-world clinical scenarios re

StakeBench: Evaluating Language Understanding Grounded in Market Commitment

SafetyDGX agent

arXiv:2605.26074v1 Announce Type: cross Abstract: Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers ha

TS-Skill: A Benchmark for Evaluating Analytical Skills in Time-Series Question Answering

Model ReleasesDGX agent

arXiv:2605.24703v1 Announce Type: cross Abstract: Large language models (LLMs) and time-series language models (TSLMs) are increasingly applied to time-series question answering (TSQA). Unlike text-on

When the Manual Lies: A Realistic Benchmark to Evaluate MCP Poisoning Attacks for LLM Agents

Model ReleasesDGX agent

arXiv:2605.24069v1 Announce Type: cross Abstract: The rise of tool-using Large Language Model (LLM) agents, standardized by protocols like the Model Context Protocol (MCP), has unlocked unprecedented

25 May 2026

Evaluating PhaseNet on Teleseismic Data with MsPASS

Local AiDGX agent

arXiv:2605.22837v1 Announce Type: cross Abstract: Numerous studies have shown that the machine-learning picker PhaseNet produces accurate P and S picks on local earthquake signals, but its performance

Speak-to-Structure: Evaluating LLMs in Open-domain Natural Language-Driven Molecule Generation

Model ReleasesDGX agent

arXiv:2412.14642v4 Announce Type: replace Abstract: Recently, Large Language Models (LLMs) have demonstrated great potential in natural language-driven molecule discovery. However, existing datasets a

22 May 2026

Evaluating Commercial AI Chatbots as News Intermediaries

Model ReleasesDGX agent

arXiv:2605.22785v1 Announce Type: new Abstract: AI chatbots are rapidly shaping how people encounter the news, yet no prior study has systematically measured how accurately these systems, with their p

Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines

Model ReleasesDGX agent

arXiv:2605.20630v1 Announce Type: new Abstract: Industrial asset operations workflows are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure

Flat-Pack Bench: Evaluating Spatio-Temporal Understanding in Large Vision-Language Models through Furniture Assembly

Model ReleasesDGX agent

arXiv:2605.21625v1 Announce Type: cross Abstract: The emergence of Large Vision-Language Models (LVLMs) has significantly advanced video understanding capabilities. However, existing benchmarks focus

21 May 2026

Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale

Model ReleasesDGX agent

arXiv:2605.20744v1 Announce Type: new Abstract: Aligning autonomous agents with human intent remains a central challenge in modern AI. A key manifestation of this challenge is reward hacking, whereby

MedicalBench: Evaluating Large Language Models Toward Improved Medical Concept Extraction

Model ReleasesDGX agent

arXiv:2605.20197v1 Announce Type: new Abstract: Medical concept extraction from electronic health records underpins many downstream applications, yet remains challenging because medically meaningful c

← Previous
1…6162636465…687
Next →