Search: “evaluation”

Temporal Dropout Risk in Learning Analytics: A Harmonized Survival Benchmark Across Dynamic and Early-Window Representations

arXiv:2604.08870v1 Announce Type: cross Abstract: Student dropout is a persistent concern in Learning Analytics, yet comparative studies frequently evaluate predictive models under heterogeneous proto

13 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulation

DGX agent

arXiv:2604.09368v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed as scalable user simulators for recommender system evaluation. Yet existing simulators per

safetyarxiv-cs-cv

13 Apr 2026

Concepts

All Papers

Auto-generated index of all papers mentioned across the wiki.

conceptspapersindex

11 Apr 2026

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

arXiv:2604.06296v1 Announce Type: cross Abstract: AI agents are increasingly deployed in real-world applications, including systems such as Manus, OpenClaw, and coding agents. Existing research has pr

AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification

model-releasesarxiv-cs-lg

arXiv:2603.13354v3 Announce Type: replace-cross Abstract: Reliable crop disease detection requires models that perform consistently across diverse acquisition conditions, yet existing evaluations ofte

An empirical study of LoRA-based fine-tuning of large language models for automated test case generation

arXiv:2604.06946v1 Announce Type: cross Abstract: Automated test case generation from natural language requirements remains a challenging problem in software engineering due to the ambiguity of requir

ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection

arXiv:2510.07927v2 Announce Type: replace Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high man

ClawBench: Can AI Agents Complete Everyday Online Tasks?

arXiv:2604.08523v1 Announce Type: new Abstract: AI agents may be able to automate your inbox, but can they automate other routine aspects of your life? Everyday online tasks offer a realistic yet unso

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge

arXiv:2510.18196v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are commonly used as evaluators in various applications, but the reliability of the outcomes remains a challenge.

safetyarxiv-cs-ai

Data Leakage in Automotive Perception: Practitioners' Insights

arXiv:2604.06899v1 Announce Type: cross Abstract: Data leakage is the inadvertent transfer of information between training and evaluation datasets that poses a subtle, yet critical, risk to the reliab

safetyarxiv-cs-lg

DosimeTron: Automating Personalized Monte Carlo Radiation Dosimetry in PET/CT with Agentic AI

arXiv:2604.06280v1 Announce Type: cross Abstract: Purpose: To develop and evaluate DosimeTron, an agentic AI system for automated patient-specific MC internal radiation dosimetry in PET/CT examination

FinTruthQA: A Benchmark for AI-Driven Financial Disclosure Quality Assessment in Investor -- Firm Interactions

arXiv:2406.12009v5 Announce Type: replace Abstract: Accurate and transparent financial information disclosure is essential for market efficiency, investor decision-making, and corporate governance. Ch

Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation

arXiv:2604.06950v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are increasingly being deployed as automated content moderators. Within this landscape, we uncover a critic

PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

arXiv:2512.23994v3 Announce Type: replace-cross Abstract: Text-to-audio-video (T2AV) generation is central to applications such as filmmaking and world modeling. However, current models often fail to

PokeGym: A Visually-Driven Long-Horizon Benchmark for Vision-Language Models

arXiv:2604.08340v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) have achieved remarkable progress in static visual understanding, their deployment in complex 3D embodied environmen

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

arXiv:2604.06409v1 Announce Type: cross Abstract: LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private.

researcharxiv-cs-ai

Spatio-Temporal Grounding of Large Language Models from Perception Streams

model-releasesarxiv-cs-ro

arXiv:2604.07592v1 Announce Type: new Abstract: Embodied-AI agents must reason about how objects move and interact in 3-D space over time, yet existing smaller frontier Large Language Models (LLMs) st

TeamLLM: A Human-Like Team-Oriented Collaboration Framework for Multi-Step Contextualized Tasks

arXiv:2604.06765v1 Announce Type: cross Abstract: Recently, multi-Large Language Model (LLM) frameworks have been proposed to solve contextualized tasks. However, these frameworks do not explicitly em

TEMPER: Testing Emotional Perturbation in Quantitative Reasoning

arXiv:2604.07801v1 Announce Type: new Abstract: Large language models are trained and evaluated on quantitative reasoning tasks written in clean, emotionally neutral language. However, real-world quer

Tools

Loving the eval track! Super relevant stuff for @clawdbench Funny pic from own of the slides @aiDotEngineer @ibragim_bad @arafatkatze and ot…

A user on X (formerly Twitter) expressed enthusiasm for an evaluation ('eval') track at an AI Engineer (@aiDotEngineer) event, noting its relevance to ClawBench — an open-source benchmarking and ev...

toolsswyx--x

9 Apr 2026

NEW paper from Microsoft Every agent benchmark has the same hidden problem: how do you know the agent actually succeeded? Microsoft research…

enterprise-aiclaudesoftware-delivery

NEW paper from Microsoft Every agent benchmark has the same hidden problem: how do you know the agent actually succeeded? Microsoft researchers introduce the Universal Verifier, which discusses lesson

model-releasesdair-ai--x

9 Apr 2026

Applications

Atlassian deploys Claude-backed Rovo Dev code review internally

Human

Atlassian reports 30.8% lower median PR cycle time in an internal evaluation of Claude-backed Rovo Dev Code Reviewer.

7 Apr 2026

3DCodeVerse: Exploring 3D Worlds as Code

arXiv:2610.22341v1 Announce Type: cross Abstract: We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises

Tutorials

Active Visual Perception: Opportunities and Challenges

arXiv:2512.03687v2 Announce Type: replace Abstract: Active visual perception enables an intelligent system to improve perception by controlling how visual evidence is acquired. However, differences in

tutorialsarxiv-cs-cv

APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

arXiv:2610.20785v1 Announce Type: new Abstract: Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental stat

Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation

arXiv:2502.13392v3 Announce Type: replace Abstract: Pioneering companies such as Waymo have deployed electric robo-taxi services in several U.S. cities. Operating such fleets requires coordinating rid

Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning

arXiv:2610.22067v1 Announce Type: new Abstract: Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. Ho

ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems

arXiv:2601.01982v2 Announce Type: replace Abstract: Large language models (LLMs) excel at natural language tasks but remain brittle in domains requiring precise logical and symbolic reasoning. Chaotic

CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering

arXiv:2610.21805v1 Announce Type: new Abstract: Multimodal objects can require different partitions under different instructions, as when products in an e-commerce catalog are grouped by color, materi

EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering

arXiv:2610.22160v1 Announce Type: cross Abstract: Test-time context optimization adapts frozen language models through textual feedback, but ordinary conditional generation provides no separate contro

Applications

Grounded Human-Attributed Description and Activity Recognition in Videos

arXiv:2610.22295v1 Announce Type: new Abstract: Understanding characteristics and activities of individuals in complex multi-person environments is crucial for real-world applications. However, existi

applicationsarxiv-cs-cv

Improving Environment-Agnostic Coding Agents: The Role of Scaffolding

arXiv:2610.22289v1 Announce Type: new Abstract: Stronger language models may increasingly construct useful procedures for themselves, reducing the advantage of developer-supplied scaffolding. We inves

In the Guise of Correction: How Framing and Cultural Adaptation Erode LLM Safety on Real-World Criminal Cases

arXiv:2610.21789v1 Announce Type: cross Abstract: Existing safety benchmarks rely largely on synthetic prompts with loosely grounded taxonomies and offer little focused analysis of severe crime catego

Noise-Space Reward Alignment for Few-Step Generative Models

arXiv:2610.21125v1 Announce Type: new Abstract: Few-step generative models produce high-quality samples in one or a few evaluations. However, steering them toward desired properties remains difficult,

safetyarxiv-cs-lg

Online Quantile Omniprediction for Proper Losses

arXiv:2610.21342v1 Announce Type: new Abstract: The task of constructing a predictor that is simultaneously accurate under multiple evaluation metrics has been popularized recently under the name omni

researcharxiv-cs-lg

Orbia: Do Generated Videos Form a Consistent 3D World?

arXiv:2610.20693v1 Announce Type: new Abstract: Video generators serve as interactive worlds for exploration and simulation, yet realistic and visually appealing frames do not necessarily depict a con

Physics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference

model-releasesarxiv-cs-lg

arXiv:2610.21833v1 Announce Type: new Abstract: Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and evaluate power flow orders of magnitude faster

PiCo: Physics-Informed Concept Probing for Geo-Foundation Models

arXiv:2610.20973v1 Announce Type: new Abstract: Geo-foundation models (GFMs) are typically evaluated by downstream task predictive performance, revealing little about which physical features they are

researcharxiv-cs-cv

Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

arXiv:2610.22041v1 Announce Type: new Abstract: World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship lear

researcharxiv-cs-ro

Qwen-Audio-Jev: A Unified Interface for Audio Decisions

arXiv:2610.21614v1 Announce Type: cross Abstract: Large audio language models (LALMs) have made substantial progress in audio understanding. However, in high-throughput applications requiring frequent

Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

arXiv:2610.20766v1 Announce Type: new Abstract: Imagined speech decoding from EEG offers a promising route toward naturalistic brain-computer interfaces. Traditional closed-set classifiers treat seman

safetyarxiv-cs-lg

Tutorials

Sensor Diagnosis for Fuel and Emissions Monitoring in Heavy-Duty Diesel Vehicles

arXiv:2610.21188v1 Announce Type: new Abstract: Faulty sensor readings can distort estimates of fuel use and carbon dioxide emissions from diesel engines. We evaluate two procedures that use relations

tutorialsarxiv-cs-lg

SSR: Structural Signal Retriever for Scalable Agent Discovery in LLM-Orchestrated Multi-Agent Systems

arXiv:2610.21113v1 Announce Type: cross Abstract: Large-scale multi-agent ecosystems require mechanisms for discovering suitable agents without exposing entire registries to a large language model (LL

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?

arXiv:2610.21253v1 Announce Type: cross Abstract: Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to

safetyarxiv-cs-ai

When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark

arXiv:2610.21555v1 Announce Type: cross Abstract: Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a

Agents

Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams

arXiv:2610.15788v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed as teams, yet they fail in characteristically social ways: information holders stay silent,

agentsarxiv-cs-ai

15 Oct 2026

Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization

arXiv:2604.25083v2 Announce Type: replace Abstract: Computer Architecture research has traditionally required the exploration of vast, highly complex design spaces through manual human effort. Large L

15 Oct 2026

Agents

AutoMARS: Automatic and Scalable Multi-Agent Large Language Model Framework for Supply Chain Risk Assessment