Search: “evaluation”

CAMI: Cost-Aware Agent-Guided Multi-Indexing for Semantic Retrieval

arXiv:2606.28365v1 Announce Type: cross Abstract: RAG ingestion pipelines frequently augment search corpus index with semantic enrichment indices (e.g., synthetic queries or summaries generated from c

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense

DGX agent

arXiv:2606.29441v1 Announce Type: cross Abstract: Inference-time safety methods for large language models have proliferated, yet no systematic comparison exists. We evaluate five defense paradigms (no

model-releasesarxiv-cs-ai

30 Jun 2026

Research

Depth Exploration for LLM Decoding

arXiv:2606.29223v1 Announce Type: new Abstract: Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate dep

researcharxiv-cs-lg

DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning

arXiv:2603.17863v2 Announce Type: replace-cross Abstract: Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and e

EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies

arXiv:2606.20092v2 Announce Type: replace Abstract: Memory remains a critical bottleneck for long-horizon robotic manipulation, as standard Vision-Language-Action (VLA) policies often fail when task-r

How Schrödinger sped up molecular discovery by 4x with Alphaevolve

model-releasesgoogle-cloud-ai

Computational chemistry researchers have traditionally faced a frustrating trade-off when simulating molecular interactions: use fast classical force fields that sacrifice precision or rely on accurat

How to Train Your Long-Context Visual Document Model

arXiv:2602.15257v3 Announce Type: replace-cross Abstract: We present the first comprehensive, large-scale study of training long-context vision language models up to 344K context, targeting long-docum

Applications

HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement in Game Engines

arXiv:2603.10604v3 Announce Type: replace Abstract: Generative models are increasingly used in video game engines to enhance the photorealism of rendered images for visual synthetic data generation an

applicationsarxiv-cs-cv

In-Vehicle Digital Twin-Based Collision Warning Framework with Sybil Attack Detection

arXiv:2606.28625v1 Announce Type: cross Abstract: Connected Vehicles (CVs) rely extensively on communication technologies to enable data-driven predictive analyses for enhancing performance and safety

Inside Genebench-Pro

Genebench-Pro appears to be a case study or tool from OpenAI focused on benchmarking or evaluating genetic/genomic analysis capabilities, likely demonstrating how OpenAI's models or tools can be appli

model-releasesopenai

Introducing GeneBench-Pro

GeneBench-Pro is a comprehensive benchmarking tool or dataset introduced by OpenAI designed to evaluate AI model performance on genomic and gene-related tasks. It likely provides standardized metrics

model-releasesopenai

Local Ai

KernelSight-LM: A Kernel-Level LLM Inference Simulator

arXiv:2606.28565v1 Announce Type: cross Abstract: As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, model

local-aiarxiv-cs-ai

LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion

arXiv:2603.14526v2 Announce Type: replace Abstract: The recent success of inference-time scaling in large language models has inspired similar explorations in video diffusion. In particular, motivated

Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models

arXiv:2601.05366v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly deployed as agents that invoke external tools through structured function calls. While recent wo

Research

Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs

arXiv:2606.29534v1 Announce Type: new Abstract: Popular ASR test sets adopt inconsistent conventions for numbers, disfluencies, entities, and casing, while standard normalizers erase the format distin

researcharxiv-cs-cl

Redefining Maritime Anomaly Detection via Equation-Grounded Synthetic Anomalies

arXiv:2606.29721v1 Announce Type: cross Abstract: Maritime anomaly detection is essential for ensuring maritime safety, security, and efficient traffic management at sea, with Automatic Identification

Reproducing FACTER: Fairness via Conformal Thresholding and Prompt Repair

arXiv:2606.28620v1 Announce Type: cross Abstract: Fayyazi et al. (2025) recently proposed FACTER, a model-agnostic framework designed to jointly enforce fairness and statistical coverage in LLM-based

RoAd-RL: A Unified Library and Benchmark for Robust Adversarial Reinforcement Learning

arXiv:2606.29867v1 Announce Type: cross Abstract: Deep Reinforcement Learning (DRL) has achieved significant success in robotics and autonomous systems, yet remains vulnerable to adversarial perturbat

Applications

See and Switch: Vision-Based Branching for Interactive Robot-Skill Programming

arXiv:2603.08057v2 Announce Type: replace-cross Abstract: Programming by demonstration (PbD) makes robot programming accessible to non-experts, but scaling it to real-world variability remains a chall

applicationsarxiv-cs-cv

Tools

See our full model rankings: http://cursor.com/evals

Cursor published a comprehensive ranking of AI models, likely evaluating them across performance metrics, capabilities, and use cases relevant to their code editor platform. The rankings help users un

toolscursor--x

Industry

Software testing startup Arato gets $10M to stop businesses deploying AI systems blind

Israeli startup Arato Software Ltd. is developing tools for developers to test and evaluate their artificial intelligence applications, and it has just gotten 10 million in seed funding to pursue that

industrysiliconangle

SWE-fficiency: Can Language Models Optimize Real-World Repositories on Real Workloads?

arXiv:2511.06090v3 Announce Type: replace-cross Abstract: Optimizing the performance of large-scale software repositories demands expertise in code reasoning and software engineering (SWE) to reduce r

TextClusterLab: An Integrated Framework for Reliable Text Clustering Studies

model-releasesarxiv-cs-lg

arXiv:2606.28328v1 Announce Type: cross Abstract: In recent years, text clustering has become a critical technique for applications including intent discovery, topic mining, and recommendation systems

The Crowded Embedding Space: A Mean-Field Mechanism for Emergent Marginalization in Retrieval-Augmented Agents

arXiv:2606.28343v1 Announce Type: cross Abstract: Retrieval-augmented generative agents rely on retrieval for grounding, yet are typically evaluated on a query-by-query basis. This isolates interactio

Agents

The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

arXiv:2606.13079v2 Announce Type: replace-cross Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical re

agentsarxiv-cs-ai

Tutorials

The Hidden Cost of Resampling: How Imbalance Correction Degrades Probability Calibration in Tree Ensembles

arXiv:2606.29720v1 Announce Type: cross Abstract: Resampling methods such as SMOTE and random under/over-sampling are standard tools for class-imbalanced classification, almost always evaluated by min

tutorialsarxiv-cs-cl

The NTNU System at the S&I Challenge 2025 SLA Open Track

model-releasesarxiv-cs-cl

arXiv:2506.05121v3 Announce Type: replace Abstract: A recent line of research on spoken language assessment (SLA) employs neural models such as BERT and wav2vec 2.0 (W2V) to evaluate speaking proficie

Agents

Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task

arXiv:2512.10359v1 Announce Type: cross Abstract: Video Question Answering (VideoQA) task serves as a critical playground for evaluating whether foundation models can effectively perceive, understand,

agentsarxiv-cs-ai

TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents

arXiv:2606.28480v1 Announce Type: cross Abstract: As large language models and harness frameworks continue to advance, agents operating in terminals are increasingly capable of performing a broader ra

ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models

arXiv:2606.28804v1 Announce Type: new Abstract: Embodied World Models (EWMs) have emerged as a scalable and risk-free paradigm for advancing embodied intelligence, enabling the safety-critical evaluat

Aloe-Vision: Robust Vision-Language Models for Healthcare

model-releasesarxiv-cs-cl

arXiv:2606.27500v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) specialized in healthcare are emerging as a promising research direction due to their potential impact in clinica

Democratic ICAI: Debating Our Way to Steering Principles from Preferences

arXiv:2606.28294v1 Announce Type: new Abstract: Preference-based alignment often struggles to capture the reasoning that underlies human judgments. Many evaluations rely on multiple interacting criter

Do Speech Emphasis Models Generalize across Languages and Emotions?

arXiv:2606.27717v1 Announce Type: cross Abstract: Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on

DysLexLens: A Low-Resource LLM Framework for Analysing Dyslexic Learners Insights from Online Forums

arXiv:2606.27619v1 Announce Type: new Abstract: Dyslexic learners increasingly use artificial intelligence (AI) tools to support reading, writing, organisation, and study-related tasks. However, their

Applications

Every Step of the Way: Video-based Parkinsonian Turning Step Counting

arXiv:2606.27918v1 Announce Type: cross Abstract: As a prominent symptom of Parkinson's disease (PD), turning impairment is evaluated through parameters such as turning angle, duration, and particular

applicationsarxiv-cs-ai

Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context

arXiv:2601.17642v2 Announce Type: replace Abstract: Safety alignment in Large Language Models is critical for healthcare; however, reliance on binary refusal boundaries often results in over-refusal o

PerturbCellRL: Verifier-Guided Reinforcement Learning for Single-Cell Perturbation Prediction

arXiv:2606.27752v1 Announce Type: new Abstract: Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond transcriptionally to interventions. While recent gen

PreferThinker: Reasoning-based Personalized Image Preference Assessment

arXiv:2511.00609v4 Announce Type: replace Abstract: Personalized image preference assessment aims to evaluate an individual user's image preferences by relying only on a small set of reference images

Productionized Fairness Measurement Under Privacy Constraints

arXiv:2606.27558v1 Announce Type: new Abstract: Fairness measurements in the form of disaggregated evaluations often rely on demographic signals that are legally constrained or culturally sensitive. R

Reasoning-Enhanced Rare-Event Prediction with Balanced Outcome Correction

arXiv:2601.16406v2 Announce Type: replace-cross Abstract: Rare-event prediction is critical in domains such as healthcare, finance, reliability engineering, customer support, aviation safety, where po

Research

Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrieval

arXiv:2606.27401v1 Announce Type: cross Abstract: Semantic code search and clone detection are essential for software development, maintenance, and reuse. This paper evaluates the effectiveness, effic

researcharxiv-cs-cl

SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion

arXiv:2606.27876v1 Announce Type: cross Abstract: Spatial intelligence is essential for low-altitude unmanned aerial vehicle (UAV) perception, collaboration, and navigation. However, existing UAV benc

SurgXBench: Explainable Vision-Language Model Benchmark for Surgery

arXiv:2505.10764v4 Announce Type: replace Abstract: Innovations in digital intelligence are transforming robotic surgery with more informed decision-making. Real-time awareness of surgical instrument

Triadic Werewolf: A Jester Role for Multi-Hop Theory of Mind in LLMs

arXiv:2606.27909v1 Announce Type: cross Abstract: Theory-of-mind evaluations of large language models typically use dyadic social-deduction games, where every observable cue points to a single hidden

Understanding Cross-Rig Generalization in Automotive Perception: a Multi-Rig Benchmark and Rig Variation Metrics

arXiv:2606.27554v1 Announce Type: new Abstract: Camera-based perception systems for autonomous driving are typically developed and evaluated using fixed sensor rigs, while real-world vehicle fleets ex

Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety

model-releasesarxiv-cs-cl

arXiv:2606.27632v1 Announce Type: new Abstract: As large language models are increasingly deployed in real-world systems, safety failures can still lead to harmful outputs and dangerous misuse. We arg

Agents

langgraph <3

langgraph <3 New category drop on the Arena: ✨AI agent frameworks✨ Agent Experience (AX) update as of 06/26: 1/ @LangChain LangGraph 2/ @vercel AI SDK 3/ @crewAIInc 4/ @mastra For context: we evaluate

agentsharrison-chase--x

27 Jun 2026

Industry

A Decision Framework for ETL Migration to Databricks