Search: “benchmark”

arXiv:2604.19572v1 Announce Type: new Abstract: As model capabilities advance, research has increasingly shifted toward long-horizon, multi-turn terminal-centric agentic tasks, where raw environment f

agentsarxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends

DGX agent

arXiv:2507.09861v2 Announce Type: replace-cross Abstract: Visually Rich Document Understanding (VRDU) has become a pivotal area of research, driven by the need to automatically interpret documents tha

agentsarxiv-cs-ai

22 Apr 2026

Are Large Language Models Economically Viable for Industry Deployment?

arXiv:2604.19342v1 Announce Type: new Abstract: Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, ente

🚨 Are we witnessing the automation of AI research? @HuggingFace just unveiled 'ML-Intern' and my mind is BLOWN 🤯 It’s an open-source pipel…

model-releasesclem-delangue--x

🚨 Are we witnessing the automation of AI research? @HuggingFace just unveiled 'ML-Intern' and my mind is BLOWN 🤯 It’s an open-source pipeline that replicates the exact daily loop of an ML researcher.

Assessing Capabilities of Large Language Models in Social Media Analytics: A Multi-task Quest

arXiv:2604.18955v1 Announce Type: cross Abstract: In this study, we present the first comprehensive evaluation of modern LLMs - including GPT-4, GPT-4o, GPT-3.5-Turbo, Gemini 1.5 Pro, DeepSeek-V3, Lla

Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset

arXiv:2604.19323v1 Announce Type: cross Abstract: Concept Bottleneck Models (CBMs) route predictions exclusively through a clinically grounded concept layer, binding interpretability to concept-label

Council Mode: Mitigating Hallucination and Bias in LLMs via Multi-Agent Consensus

arXiv:2604.02923v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs), particularly those employing Mixture-of-Experts (MoE) architectures, have achieved remarkable capabilities acros

Decompose, Structure, and Repair: A Neuro-Symbolic Framework for Autoformalization via Operator Trees

arXiv:2604.19000v1 Announce Type: cross Abstract: Statement autoformalization acts as a critical bridge between human mathematics and formal mathematics by translating natural language problems into f

Deep sprite-based image models: An analysis

arXiv:2604.19480v1 Announce Type: new Abstract: While foundation models drive steady progress in image segmentation and diffusion algorithms compose always more realistic images, the seemingly simple

Disparities In Negation Understanding Across Languages In Vision-Language Models

arXiv:2604.18942v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit affirmation bias: a systematic tendency to select positive captions ('X is present') even when the correct descrip

Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture The Flag Challenges

arXiv:2604.19354v1 Announce Type: new Abstract: Large Language Model (LLM) agents are increasingly proposed for autonomous cybersecurity tasks, but their capabilities in realistic offensive settings r

DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning

arXiv:2604.18829v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved impressive performance on visual perception and reasoning tasks with RGB imagery, yet they remain

EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion

arXiv:2604.18790v1 Announce Type: new Abstract: Depth completion from sparse LiDAR measurements and corresponding RGB images is a prerequisite for accurate 3D perception in robotic systems. Existing m

Applications

Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling

arXiv:2604.18753v1 Announce Type: cross Abstract: An active challenge in developing multimodal machine learning (ML) models for healthcare is handling missing modalities during training and deployment

applicationsarxiv-cs-ai

LoViF 2026 Challenge on Real-World All-in-One Image Restoration: Methods and Results

arXiv:2604.19445v1 Announce Type: new Abstract: This paper presents a review for the LoViF Challenge on Real-World All-in-One Image Restoration. The challenge aimed to advance research on real-world a

OmniGen2: Towards Instruction-Aligned Multimodal Generation

arXiv:2506.18871v4 Announce Type: replace-cross Abstract: In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generatio

OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

arXiv:2604.18827v1 Announce Type: cross Abstract: Scaling data and artificial neural networks has transformed AI, driving breakthroughs in language and vision. Whether similar principles apply to mode

Once Correct, Still Wrong: Counterfactual Hallucination in Multilingual Vision-Language Models

arXiv:2602.05437v2 Announce Type: replace Abstract: Vision-language models (VLMs) can achieve high accuracy while still accepting culturally plausible but visually incorrect interpretations. Existing

ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models

arXiv:2509.15435v2 Announce Type: replace-cross Abstract: Large Vision-Language Models (LVLMs) exhibit strong multimodal capabilities but remain vulnerable to hallucinations from intrinsic errors and

Protecting Bystander Privacy via Selective Hearing in Audio LLMs

arXiv:2512.06380v3 Announce Type: replace-cross Abstract: Audio Large language models (LLMs) are increasingly deployed in the real world, where they inevitably capture speech from unintended nearby by

Realistic Handwritten Multi-Digit Writer (MDW) Number Recognition Challenges

arXiv:2512.00676v2 Announce Type: replace Abstract: Isolated digit classification has served as a motivating problem for decades of machine learning research. In real settings, numbers often occur as

ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation

arXiv:2604.19144v1 Announce Type: new Abstract: Recent years have witnessed growing interest in applying Large Reasoning Models (LRMs) to Machine Translation (MT). Existing approaches predominantly ad

SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

arXiv:2604.19638v1 Announce Type: new Abstract: Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address saf

SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion

arXiv:2510.17925v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at code-related tasks but often struggle in realistic software repositories, where project-specific APIs an

StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis

arXiv:2510.10074v2 Announce Type: replace Abstract: Effective incident management in large-scale IT systems relies on troubleshooting guides (TSGs), but their manual execution is slow and error-prone.

StomaD2: An All-in-One System for Intelligent Stomatal Phenotype Analysis via Diffusion-Based Restoration Detection Network

arXiv:2604.18632v1 Announce Type: new Abstract: Stomata play a crucial role in regulating plant physiological processes and reflecting environmental responses. However, accurate and high-throughput st

researcharxiv-cs-cv

Talking to a Know-It-All GPT or a Second-Guesser Claude? How Repair reveals unreliable Multi-Turn Behavior in LLMs

arXiv:2604.19245v1 Announce Type: cross Abstract: Repair, an important resource for resolving trouble in human-human conversation, remains underexplored in human-LLM interaction. In this study, we inv

Tools

That's from this ChatGPT conversation - https://chatgpt.com/share/69e90ba5-7760-83e8-9e09-f06e25a90930 - my prompt was 'Do a where's Waldo s…

That's from this ChatGPT conversation - https://chatgpt.com/share/69e90ba5-7760-83e8-9e09-f06e25a90930 - my prompt was 'Do a where's Waldo style image but it's where is the raccoon holding a ham radio

toolssimon-willison--x

Virtual boundary integral neural network for three-dimensional exterior acoustic problems

arXiv:2604.18636v1 Announce Type: cross Abstract: This paper presents a virtual boundary integral neural network (VBINN) for exterior acoustic problems in three dimensions. The method introduces a vir

researcharxiv-cs-lg

VISTA: Verification In Sequential Turn-based Assessment

arXiv:2510.27052v5 Announce Type: replace Abstract: Hallucination--defined here as generating statements unsupported or contradicted by available evidence or conversational context--remains a major ob

researcharxiv-cs-cl

A Survey of Spatial Memory Representations for Efficient Robot Navigation

arXiv:2604.16482v1 Announce Type: new Abstract: As vision-based robots navigate larger environments, their spatial memory grows without bound, eventually exhausting computational resources, particular

Agents

Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity

arXiv:2604.17609v1 Announce Type: new Abstract: LLM-based agents are assumed to integrate environmental observations into their reasoning: discovering highly relevant but unexpected information should

agentsarxiv-cs-cl

Alexandria: A Multi-Domain Dialectal Arabic Machine Translation Dataset for Culturally Inclusive and Linguistically Diverse LLMs

arXiv:2601.13099v2 Announce Type: replace Abstract: Arabic is a highly diglossic language where most daily communication occurs in regional dialects rather than Modern Standard Arabic (MSA). Despite t

Back to Repair: A Minimal Denoising Network for Time Series Anomaly Detection

model-releasesarxiv-cs-lg

arXiv:2604.17388v1 Announce Type: new Abstract: We introduce JuRe (Just Repair), a minimal denoising network for time series anomaly detection that exposes a central finding: architectural complexity

BridgeEQA: Virtual Embodied Agents for Real Bridge Inspections

arXiv:2511.12676v2 Announce Type: replace Abstract: Deploying embodied agents that can answer questions about their surroundings in realistic real-world settings remains difficult, partly due to the s

Local Ai

Bridging the Culture Gap: A Framework for LLM-Driven Socio-Cultural Localization of Math Word Problems in Low-Resource Languages

arXiv:2508.14913v4 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated significant capabilities in solving mathematical problems expressed in natural language. However, mul

local-aiarxiv-cs-cl

Can Large Language Models Understand Context?

model-releasesapple-ml-research

Understanding context is key to understanding human language, an ability which Large Language Models (LLMs) have been increasingly seen to demonstrate to an impressive extent. However, though the eval

Safety

COSEARCH: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search

arXiv:2604.17555v1 Announce Type: cross Abstract: Agentic search -- the task of training agents that iteratively reason, issue queries, and synthesize retrieved information to answer complex questions

safetyarxiv-cs-cl

Safety

CrossFlowDG: Bridging the Modality Gap with Cross-modal Flow Matching for Domain Generalization

arXiv:2604.16892v1 Announce Type: new Abstract: Domain generalization (DG) aims to maintain performance under domain shift, which in computer vision appears primarily as stylistic variations that caus

safetyarxiv-cs-cv

CWT-Enhanced Vibration Sensing With Spatial Fault Localization Using YOLO

arXiv:2509.03070v4 Announce Type: replace-cross Abstract: This letter presents a CWT-enhanced vibration sensing framework for bearing fault monitoring through spatial localization on time-frequency sp

researcharxiv-cs-cv

Document-as-Image Representations Fall Short for Scientific Retrieval

arXiv:2604.18508v1 Announce Type: cross Abstract: Many recent document embedding models are trained on document-as-image representations, embedding rendered pages as images rather than the underlying

EasyVideoR1: Easier RL for Video Understanding

arXiv:2604.16893v1 Announce Type: new Abstract: Reinforcement learning from verifiable rewards (RLVR) has demonstrated remarkable effectiveness in improving the reasoning capabilities of large languag

EdgeVTP: Exploration of Latency-efficient Trajectory Prediction for Edge-based Embedded Vision Applications

arXiv:2604.16783v1 Announce Type: new Abstract: Vehicle trajectory prediction is central to highway perception, but deployment on roadside edge devices necessitates bounded, deterministic end-to-end l

researcharxiv-cs-cv

From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration

arXiv:2604.16462v1 Announce Type: new Abstract: High-resolution Multimodal Large Language Models (MLLMs) face prohibitive computational costs during inference due to the explosion of visual tokens. Ex

GoCoMA: Hyperbolic Multimodal Representation Fusion for Large Language Model-Generated Code Attribution

arXiv:2604.16377v1 Announce Type: new Abstract: Large Language Models (LLMs) trained on massive code corpora are now increasingly capable of generating code that is hard to distinguish from human-writ

researcharxiv-cs-cl

Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy

arXiv:2510.22215v2 Announce Type: replace-cross Abstract: Retrieval over visually rich documents is essential for tasks such as legal discovery, scientific search, and enterprise knowledge management.

InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts

arXiv:2509.10813v3 Announce Type: replace Abstract: The advancement of Embodied AI heavily relies on large-scale, simulatable 3D scene datasets characterized by scene diversity and realistic layouts.

LLaMA-XR: A Novel Framework for Radiology Report Generation using LLaMA and QLoRA Fine Tuning

arXiv:2506.03178v2 Announce Type: replace-cross Abstract: Automated radiology report generation holds significant potential to reduce radiologists' workload and enhance diagnostic accuracy. However, g