Search: “evaluation”

DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable

arXiv:2608.00548v1 Announce Type: new Abstract: Recent image-generation models and multimodal agents can produce high-quality visuals for increasingly complex visual communication tasks. Yet their ras

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Foveated Probes Recover Localized Binding Information in Vision Foundation Models

DGX agent

arXiv:2608.00726v1 Announce Type: new Abstract: Frozen vision foundation models are commonly evaluated through a single global image embedding, but this interface can conflate missing information with

researcharxiv-cs-cv

4 Aug 2026

Gaokerena: A Small Persian Medical Language Model Family

arXiv:2608.00932v1 Announce Type: new Abstract: The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused

GEOID-Flood: A Large-Scale Multi-Modal Benchmark Dataset for Flood Segmentation

arXiv:2608.02315v1 Announce Type: new Abstract: Geospatial foundation models aim to learn representations that transfer across regions and sensors, yet evaluating them on specific tasks requires large

Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models

arXiv:2608.00144v1 Announce Type: cross Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind bas

researcharxiv-cs-cl

Long-Horizon Embodied Decision-Making via Multimodal Memory Compression

arXiv:2608.01456v1 Announce Type: cross Abstract: Agents are increasingly expected to act not only as task executors, but also as decision-makers on behalf of human users. This shift requires agents t

LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning

arXiv:2608.01328v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart un

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

arXiv:2608.00677v1 Announce Type: new Abstract: AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-model i

Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality

arXiv:2608.01133v1 Announce Type: new Abstract: Evaluating Multi-Agent Reinforcement Learning (MARL) policies in autonomous driving fundamentally relies on extrinsic statistical indicators (e.g., rewa

safetyarxiv-cs-lg

Pruned BPE: Post-training Visibility Pruning and Token Reallocation for Byte Pair Encoding

arXiv:2608.00837v1 Announce Type: new Abstract: Byte Pair Encoding (BPE) is widely used for subword tokenization, but standard BPE exposes every learned merge token to the downstream model, including

researcharxiv-cs-cl

Applications

Reassessing the Feasibility of PPG-Based Non-Invasive Blood Glucose Level Estimation

arXiv:2608.01820v1 Announce Type: cross Abstract: Non-invasive blood glucose level (BGL) estimation from photoplethysmography (PPG) holds great promise for wearable health monitoring, but results acro

applicationsarxiv-cs-lg

RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?

arXiv:2608.02039v1 Announce Type: new Abstract: Remote-sensing videos enable real-time observation of changes in target attributes, short-term activities, and scene evolution. They record motion, acti

Stochastic Sequential Search in Very-High-Dimensional Feature Selection

arXiv:2608.01502v1 Announce Type: cross Abstract: Sequential subset search -- forward selection with floating backtracking and its descendants -- remains the quality reference in feature selection, bu

researcharxiv-cs-cv

Local Ai

Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations

arXiv:2608.00561v1 Announce Type: cross Abstract: Vision-language models (VLMs) process image patches and text tokens in a shared residual stream, but the local geometry through which the two modaliti

local-aiarxiv-cs-cl

Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms

arXiv:2608.01004v1 Announce Type: new Abstract: Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to thei

WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model

arXiv:2608.01845v1 Announce Type: cross Abstract: Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse

researcharxiv-cs-cv

Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

arXiv:2607.28661v1 Announce Type: new Abstract: Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding

Autonomous Repair for Multi-Agent Systems via Monte-Carlo Tree Search

arXiv:2607.29055v1 Announce Type: cross Abstract: Multi-agent systems (MAS) are increasingly deployed to solve complex tasks. In case of incorrect or unsatisfactory outputs, users have to manually loc

Benchmarks Are Not Validation: A System-Level View of Financial LLM Applications

arXiv:2607.28840v1 Announce Type: new Abstract: Large language models are increasingly deployed in financial applications that combine retrieval, proprietary data, tool use, orchestration logic, monit

Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery

arXiv:2603.03322v2 Announce Type: replace-cross Abstract: Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery. However, rig

Can Zero-Shot LLMs Predict Child Malnutrition? A Fairness and Temporal Robustness Study

arXiv:2607.29082v1 Announce Type: new Abstract: Child malnutrition remains a major public health challenge in low- and middle-income countries, particularly in South Asia, where early identification o

safetyarxiv-cs-cl

Detecting AI-Generated Videos with Spiking Neural Networks

arXiv:2605.05895v2 Announce Type: replace-cross Abstract: Modern AI-generated videos are photorealistic at the single-frame level, leaving inter-frame dynamics as the main remaining axis for detection

Language Models Agree With Each Other, Not With Readers

arXiv:2607.29274v1 Announce Type: cross Abstract: Claims that language models homogenise are usually measured against human judgements collected for the study, which makes the human side an artifact o

Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation

arXiv:2607.28645v1 Announce Type: cross Abstract: Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshot

OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systems

arXiv:2607.28629v1 Announce Type: new Abstract: The rapid transition from reactive large language models (LLMs) to persistent, action-capable systems has exposed critical gaps in the architectural und

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

arXiv:2607.28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning. These developments have accelerated the

safetyarxiv-cs-ai

Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

arXiv:2607.28665v1 Announce Type: cross Abstract: Automated driving systems (ADSs) are becoming ubiquitous. Future Software Defined Vehicles (SDVs) may be able to run multiple ADSs, both native and af

safetyarxiv-cs-ai

The Grokked Illusion: True Equilibrium Mitigates Catastrophic Forgetting

arXiv:2607.29503v1 Announce Type: new Abstract: While neural networks are typically evaluated by their training and test performance, these metrics do not reveal how robust a learned representation is

Why It Hurts: Identifying the Drivers of Negative Thoughts in Emotional Support Conversations

arXiv:2607.28648v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for emotional support tasks, such as negative thought reframing. This task relies on modifying cogn

A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models

arXiv:2607.26102v1 Announce Type: cross Abstract: Mathematical chain of thought (CoT) evaluation is commonly reduced to whether the final answer matches a reference. This conflates producing a correct

researcharxiv-cs-ai

A Systems Engineering Framework for Vision-Language-Enabled UAV Triage and Disaster Response

arXiv:2607.27597v1 Announce Type: new Abstract: Recent advances in Vision Language Models (VLMs) have created new opportunities for disaster response, where responders must interpret large volumes of

safetyarxiv-cs-ro

All oneshots from Kimi-K3, looks better than opus4.8.

model-releasesr-localllama

I've ran Kimi-k3 through 34 oneshot prompts and evaluated the generated htmls, screenshots and gifs using sonnet 4.6. It came out to be better than opus4.8 from the evals. Kimi K3: https://oneshotlm.c

An Efficient Solution to the 2D Visibility Problem in Cartesian Grid Maps and its Application in Heuristic Path Planning

arXiv:2403.06494v2 Announce Type: replace-cross Abstract: This paper introduces a novel, lightweight method to solve the visibility problem for 2D grids. The proposed method evaluates the existence of

researcharxiv-cs-ro

Local Ai

Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer

arXiv:2607.17100v2 Announce Type: replace-cross Abstract: Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged

local-aiarxiv-cs-ai

Benchmarking Foundation and Large Language Models for Few-Shot Medical Image Segmentation

arXiv:2607.27856v1 Announce Type: new Abstract: Few-shot medical image segmentation (FS-MIS) aims to segment novel regions of interest (ROIs) from a few annotated support examples. Despite rapid progr

Beyond Feeling Better: Capability-Sustaining Emotional Dialogue as a Longitudinal Research Paradigm

arXiv:2607.27851v1 Announce Type: new Abstract: Emotional dialogue research includes two influential strategy traditions. Empathetic dialogue prioritizes understanding a speaker's emotional experience

safetyarxiv-cs-cl

Applications

Calibrate Globally, Measure Everywhere: Scaling LLM-Based Prevalence Measurement Across A/B Experiments

arXiv:2602.16111v2 Announce Type: replace-cross Abstract: Online media platforms track the share of impressions associated with content attributes, or prevalence, to evaluate trade-offs and set guardr

applicationsarxiv-cs-ai

Dimensionality and Measurement Precision in HLE's Multiple-Choice Subset

arXiv:2607.27420v1 Announce Type: cross Abstract: Humanity's Last Exam (HLE) is widely used to evaluate frontier language models. HLE organizes its questions into eight subject-domain categories, whos

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

arXiv:2607.28319v1 Announce Type: new Abstract: This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias

IGME: Efficient Chained Method Ensemble for Transferable Semantic Segmentation Attacks

arXiv:2607.27465v1 Announce Type: new Abstract: Semantic segmentation models are vulnerable to transferable adversarial perturbations, yet evaluating transfer attacks on dense prediction models can be

researcharxiv-cs-cv

KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation

arXiv:2607.27231v1 Announce Type: cross Abstract: Large language models (LLMs) have significantly increased the demand for efficient accelerator kernels, but kernel development remains a highly specia

Tutorials

MetaRank: Task-Aware Metric Selection for Model Transferability Estimation

arXiv:2511.21007v2 Announce Type: replace Abstract: Selecting an appropriate pre-trained source model is a critical, yet computationally expensive, task in transfer learning. Model Transferability Est

tutorialsarxiv-cs-cv

Metrics vs Surveys: An Analysis for Human-Aligned Benchmarking in Social Robot Navigation

arXiv:2510.02941v2 Announce Type: replace Abstract: Social, also called human-aware, navigation is a key challenge for integrating mobile robots into human environments. The evaluation of such systems

safetyarxiv-cs-ro

Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory

arXiv:2607.28437v1 Announce Type: new Abstract: Molecular optimization is commonly performed under a limited oracle budget, which makes deciding what to evaluate as important as deciding what to gener

Psych-ECA: A Reproducible Semi-Synthetic Benchmark for Synthetic Control Arms in Longitudinal Psychiatry

arXiv:2607.27224v1 Announce Type: cross Abstract: External and synthetic control arms (ECAs) are entering psychiatric drug development, but the field lacks a benchmark that evaluates the properties re

Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding

arXiv:2607.27826v1 Announce Type: cross Abstract: Recent advances in sign language (SL) understanding (SLU) have led to remarkable progress in tasks such as continuous SL recognition and SL translatio

State-Dependent Safety Failures in Multi-Turn Language Model Interaction

arXiv:2603.15684v2 Announce Type: replace-cross Abstract: Safety alignment in large language models is typically evaluated under isolated queries, yet real-world use is inherently multi-turn. Although

safetyarxiv-cs-ai

This is both a real incident (in that the AI really did get unauthorized access to real systems) and also something it was (sort of) prompte…

model-releasesethan-mollick--x

This is both a real incident (in that the AI really did get unauthorized access to real systems) and also something it was (sort of) prompted to do. In a review of our cybersecurity evaluations, we fo