Search: “benchmark”

QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering

arXiv:2604.24052v1 Announce Type: cross Abstract: Video-to-text summarization remains underexplored in terms of comprehensive evaluation methods. Traditional n-gram overlap-based metrics and recent la

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Quantifying and Mitigating Self-Preference Bias of LLM Judges

DGX agent

arXiv:2604.22891v1 Announce Type: cross Abstract: LLM-as-a-Judge has become a dominant approach in automated evaluation systems, playing critical roles in model alignment, leaderboard construction, qu

safetyarxiv-cs-ai

28 Apr 2026

ResAF-Net: An Anchor-Free Attention-Based Network for Tree Detection and Agricultural Mapping in Palestine

arXiv:2604.23653v1 Announce Type: cross Abstract: Reliable agricultural data is essential for food security, land-use planning, and economic resilience, yet in Palestine, such data remains difficult t

Resolution scaling governs DINOv3 transfer performance in chest radiograph classification

arXiv:2510.07191v3 Announce Type: replace-cross Abstract: Self-supervised learning (SSL) has improved visual representation learning, but its value in chest radiography remains uncertain. DINOv3 exten

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning

arXiv:2604.23747v1 Announce Type: cross Abstract: Recent mixed-policy optimization methods for LLM reasoning that interleave or blend supervised and reinforcement learning signals report improvements

SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing

arXiv:2604.23392v1 Announce Type: new Abstract: Refereeing is vital in sports, where fair, accurate, and explainable decisions are fundamental. While intelligent assistant technologies are being widel

SWE-QA: Can Language Models Answer Repository-level Code Questions?

arXiv:2509.14635v2 Announce Type: replace Abstract: Understanding and reasoning about entire software repositories is an essential capability for intelligent software engineering tools. While existing

Tube Diffusion Policy: Reactive Visual-Tactile Policy Learning for Contact-rich Manipulation

model-releasesarxiv-cs-ro

arXiv:2604.23609v1 Announce Type: new Abstract: Contact-rich manipulation is central to many everyday human activities, requiring continuous adaptation to contact uncertainty and external disturbances

Unstable Rankings in Bayesian Deep Learning Evaluation

arXiv:2604.23102v1 Announce Type: new Abstract: Standard evaluations of Bayesian deep learning methods assume that metric estimates are reliable, but we show this assumption fails under data scarcity.

researcharxiv-cs-lg

xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models

arXiv:2509.13021v2 Announce Type: replace-cross Abstract: This work introduces xOffense, an AI-driven, multi-agent penetration testing framework that shifts the process from labor-intensive, expert-dr

Calibrating Behavioral Parameters with Large Language Models

arXiv:2602.01022v2 Announce Type: replace-cross Abstract: Behavioral parameters such as loss aversion, herding, and extrapolation are central to asset pricing models but remain difficult to measure re

ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding

arXiv:2604.22333v1 Announce Type: cross Abstract: Rapid situational awareness is critical in post-disaster response. While remote sensing damage assessment is evolving from pixel-level change detectio

Contexts are Never Long Enough: Structured Reasoning for Scalable Question Answering over Long Document Sets

arXiv:2604.22294v1 Announce Type: cross Abstract: Real-world document question answering is challenging. Analysts must synthesize evidence across multiple documents and different parts of each documen

Different Strokes for Different Folks: Writer Identification for Historical Arabic Manuscripts

arXiv:2604.22515v1 Announce Type: new Abstract: Handwritten Arabic manuscripts preserve the Arab world's intellectual and cultural heritage, and writer identification supports provenance, authenticity

DimABSA: Building Multilingual and Multidomain Datasets for Dimensional Aspect-Based Sentiment Analysis

arXiv:2601.23022v3 Announce Type: replace Abstract: Aspect-Based Sentiment Analysis (ABSA) focuses on extracting sentiment at a fine-grained aspect level and has been widely applied across real-world

Distance-Misaligned Training in Graph Transformers and Adaptive Graph-Aware Control

arXiv:2604.22413v1 Announce Type: cross Abstract: Graph Transformers can mix information globally, but this flexibility also creates failure modes: some tasks require long-range communication while ot

Local Ai

Feedback Over Form: Why Execution Feedback Matters More Than Pipeline Topology in 1-3B Code Generation

arXiv:2604.21950v1 Announce Type: cross Abstract: Small language models (1-3B) are practical to run locally, but individually limited on harder code generation tasks. We ask whether composing them int

local-aiarxiv-cs-ai

From Global to Local: Rethinking CLIP Feature Aggregation for Person Re-Identification

arXiv:2604.22190v1 Announce Type: cross Abstract: CLIP-based person re-identification (ReID) methods aggregate spatial features into a single global exttt{[CLS]} token optimized for image-text alignme

Graph-to-Vision: Multi-graph Understanding and Reasoning using Vision-Language Models

arXiv:2503.21435v3 Announce Type: replace Abstract: Recent advances in Vision-Language Models (VLMs) have shown promising capabilities in interpreting visualized graph data, offering a new perspective

How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

arXiv:2604.22750v1 Announce Type: new Abstract: The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that requi

How Popsa used Amazon Nova to inspire customers with personalised title suggestions

model-releasesaws-ml-blog

In this post, we share how we applied Amazon Bedrock and the Amazon Nova family of models to reimagine our Title Suggestion feature. By combining metadata, computer vision, and retrieval-augmented gen

Measuring and Mitigating Persona Distortions from AI Writing Assistance

arXiv:2604.22503v1 Announce Type: new Abstract: Hundreds of millions of people use artificial intelligence (AI) for writing assistance. Here, we evaluated how AI writing assistance distorts writer per

researcharxiv-cs-cl

Agents

Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results

arXiv:2604.21965v1 Announce Type: new Abstract: Recent work has used LLM agents to reproduce empirical social science results with access to both the data and code. We broaden this scope by asking: Ca

agentsarxiv-cs-ai

Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review

arXiv:2506.14831v3 Announce Type: replace Abstract: With the emergence of powerful data-driven methods in human trajectory prediction (HTP), gaining a finer understanding of multi-agent interactions l

Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores

arXiv:2604.22063v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly utilized in clinical reasoning and risk assessment. However, their interpretive reliability in critical

Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity

arXiv:2604.22597v1 Announce Type: new Abstract: Recent advancements in large language models have led to significant improvements across various tasks, including mathematical reasoning, which is used

researcharxiv-cs-ai

Applications

The problem is that the incentives push for 'more' over 'better' Paper: https://pubsonline.informs.org/doi/full/10.1287/orsc.2026.ed.v37.n3

applicationsethan-mollick--x

This paper examines how organizational incentive structures often prioritize quantitative growth and output volume over quality improvements, creating misaligned motivations that can undermine long-te

Towards Temporal Compositional Reasoning in Long-Form Sports Videos

arXiv:2604.22226v1 Announce Type: new Abstract: Sports videos are a challenging domain for multimodal understanding because they involve complex and dynamic human activities. Despite rapid progress in

TS-Arena -- A Live Forecast Pre-Registration Platform

arXiv:2512.20761v3 Announce Type: replace-cross Abstract: Time Series Foundation Models (TSFMs) are transforming the field of forecasting. However, evaluating them on historical data is increasingly d

V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models

arXiv:2504.06148v3 Announce Type: replace Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual-text processing. However, existi

researcharxiv-cs-cv

THIS GUY LOST $200 IN ONE DAY BECAUSE THE STRING 'HERMES.md' WAS IN HIS GIT COMMITS HERMES.md is a real convention used in AI agent projects…

model-releasesjeremy-howard--x

THIS GUY LOST 200 IN ONE DAY BECAUSE THE STRING 'HERMES.md' WAS IN HIS GIT COMMITS HERMES.md is a real convention used in AI agent projects. it's a system prompt specification file. not some obscure e

26 Apr 2026

Comprehensive Multilingual Text Rendering： Better glyph accuracy, more consistent typography, and cleaner layouts even in complex compositio…

Comprehensive Multilingual Text Rendering： Better glyph accuracy, more consistent typography, and cleaner layouts even in complex compositions. It handles mixed-language scenarios more gracefully as w

model-releasesqwen--x

25 Apr 2026

Applications

I think that academia has not absorbed the fact that AI agents are now good enough to independently reconstruct complex papers without acces…

applicationsethan-mollick--x

I think that academia has not absorbed the fact that AI agents are now good enough to independently reconstruct complex papers without access to code or the papers themselves; just the methods & data.

25 Apr 2026

AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models

arXiv:2604.20996v1 Announce Type: new Abstract: How can language learning systems be developed for languages that lack sufficient training resources? This challenge is increasingly faced by developers

Agents

AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use

arXiv:2604.21590v1 Announce Type: new Abstract: Modern industrial applications increasingly demand language models that act as agents, capable of multi-step reasoning and tool use in real-world settin

agentsarxiv-cs-cl

DeepSeek V4 - almost on the frontier, a fraction of the price

model-releasessimon-willison

Chinese AI lab DeepSeek's last model release was V3.2 (and V3.2 Speciale) last December. They just dropped the first of their hotly anticipated V4 series in the shape of two preview models, DeepSeek-V

DenoiseRank: Learning to Rank by Diffusion Models

arXiv:2604.20852v1 Announce Type: cross Abstract: Learning to rank (LTR) is one of the core tasks in Machine Learning. Traditional LTR models have made great progress, but nearly all of them are imple

Dr. Assistant: Enhancing Clinical Diagnostic Inquiry via Structured Diagnostic Reasoning Data and Reinforcement Learning

arXiv:2601.13690v2 Announce Type: replace Abstract: Clinical Decision Support Systems (CDSSs) provide reasoning and inquiry guidance for physicians, yet they face notable challenges, including high ma

Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI

arXiv:2604.20972v1 Announce Type: new Abstract: Content moderation systems are typically evaluated by measuring agreement with human labels. In rule-governed environments this assumption fails: multip

Evaluating AI Meeting Summaries with a Reusable Cross-Domain Pipeline

arXiv:2604.21345v1 Announce Type: new Abstract: We present a reusable evaluation pipeline for generative AI applications, instantiated for AI meeting summaries and released with a public artifact pack

Fine-Tuning Regimes Define Distinct Continual Learning Problems

model-releasesarxiv-cs-lg

arXiv:2604.21927v1 Announce Type: new Abstract: Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benc

From Codebooks to VLMs: Evaluating Automated Visual Discourse Analysis for Climate Change on Social Media

arXiv:2604.21786v1 Announce Type: new Abstract: Social media platforms have become primary arenas for climate communication, generating millions of images and posts that - if systematically analysed -

Applications

Grok Voice is used by @Starlink

Grok Voice is used by @Starlink Introducing Grok Voice Think Fast 1.0 A state-of-the-art voice model built for complex, multi-step workflows with snappy responses and high accuracy. It takes the top s

applicationselon-musk--x

How VLAs (Really) Work In Open-World Environments

arXiv:2604.21192v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) have been extensively used in robotics applications, achieving great success in various manipulation problems. Mo

Identifying Bias in Machine-generated Text Detection

arXiv:2512.09292v2 Announce Type: replace-cross Abstract: The meteoric rise in text generation capability has been accompanied by parallel growth in interest in machine-generated text detection: the c

KGiRAG: An Iterative GraphRAG Approach for Responding Sensemaking Queries

arXiv:2604.20859v1 Announce Type: cross Abstract: Recent literature highlights the potential of graph-based approaches within large language model (LLM) retrieval-augmented generation (RAG) pipelines

researcharxiv-cs-ai

MATRAG: Multi-Agent Transparent Retrieval-Augmented Generation for Explainable Recommendations

arXiv:2604.20848v1 Announce Type: cross Abstract: Large Language Model (LLM)-based recommendation systems have demonstrated remarkable capabilities in understanding user preferences and generating per

Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review

arXiv:2603.18740v2 Announce Type: replace-cross Abstract: Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, rangi