Search: “evaluation”

Artificial Analysis needs to address HiDream-01 Benchmarks

local-air-stablediffusion

HiDream-O1-Image debuted at #8 in the Artificial Analysis Text to Image Arena , and the Reddit post likely discusses whether this benchmark inclusion and ranking adequately reflect HiDream-01's perfor

10 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

The future of Math is mathematicians and AI agents working together. Very pleased to introduce @GoogleDeepMind's AI co-mathematician: a mult…

DGX agent

The future of Math is mathematicians and AI agents working together. Very pleased to introduce @GoogleDeepMind's AI co-mathematician: a multi-agent system designed to actively collaborate with human e

agentsgary-marcus--x

8 May 2026

Conceptors for Semantic Steering

arXiv:2605.04980v1 Announce Type: cross Abstract: Activation-based steering provides control of LLM behavior at inference time, but the dominant paradigm reduces each concept to a single direction who

Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models

arXiv:2605.04830v1 Announce Type: new Abstract: Diffusion models undergo a phase transition in a critical time window during generation dynamics, with two complementary diagnoses of criticality. The s

local-aiarxiv-cs-lg

cotomi Act: Learning to Automate Work by Watching You

arXiv:2605.03231v1 Announce Type: new Abstract: What if a browser agent could learn your work simply by watching you do it? We present cotomi Act, a browser-based computer-using agent that combines re

Demystifying Manifold Constraints in LLM Pre-training

arXiv:2605.04418v1 Announce Type: new Abstract: The empirical success of large language model (LLM) pre-training relies heavily on heuristic stabilization techniques, such as explicit normalization la

researcharxiv-cs-lg

Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework

arXiv:2605.04079v1 Announce Type: new Abstract: Early and reliable detection of Alzheimer's disease (AD) is crucial for timely clinical intervention and improved patient management. It also supports t

researcharxiv-cs-lg

Event-Based Early Warning of Vineyard Disease Risk from Environmental Time Series

arXiv:2605.04548v1 Announce Type: new Abstract: Accurate early warning of vineyard disease risk from environmental observations is essential for timely intervention and more sustainable crop protectio

Gray-Box Poisoning of Continuous Malware Ingestion Pipelines

arXiv:2605.04698v1 Announce Type: cross Abstract: Modern malware detection pipelines rely on continuous data ingestion and machine learning to counter the high volume of novel threats. This work inves

Tutorials

HistoMet: A Pan-Cancer Deep Learning Framework for Prognostic Prediction of Metastatic Progression and Site Tropism from Primary Tumor Histopathology

arXiv:2602.07608v2 Announce Type: replace Abstract: Metastatic Progression remains the leading cause of cancer-related mortality, yet predicting whether a primary tumor will metastasize and where it w

tutorialsarxiv-cs-cv

Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity

arXiv:2605.05071v1 Announce Type: cross Abstract: Millimeter-wave (mmWave) frequencies promise multi-gigabit connectivity for vehicle-to-everything (V2X) networks, but face challenges in terms of seve

safetyarxiv-cs-cv

LoViF 2026 The First Challenge on Holistic Quality Assessment for 4D World Model (PhyScore)

model-releasesarxiv-cs-cv

arXiv:2605.05187v1 Announce Type: new Abstract: This paper reports on the LoViF 2026 PhyScore challenge, a competition on holistic quality assessment of world-model-generated videos across both 2D and

Neural-Guided Domain Restriction to Accelerate Pseudospectra Computation for Structured Non-normal Banded Matrices

arXiv:2605.04550v1 Announce Type: cross Abstract: Computing pseudospectra of non-normal matrices is essential for understanding the stability and transient behavior of dynamical systems. Such analysis

researcharxiv-cs-lg

Not All That Is Fluent Is Factual: Investigating Hallucinations of Large Language Models in Academic Writing

arXiv:2605.04171v1 Announce Type: new Abstract: Large Language models (LLMs) show extraordinary abilities, but they are still prone to hallucinations, especially when we use them for generating Academ

Open-Source Image Editing Models Are Zero-Shot Vision Learners

arXiv:2605.04566v1 Announce Type: cross Abstract: Recent studies have shown that large generative models can solve vision tasks they were not explicitly trained for. However, existing evidence relies

Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs

arXiv:2605.04665v1 Announce Type: new Abstract: When the substantive content of a request is rewritten, do large language models still answer in the format the original task asked for? We find that th

Personalized Spiking Neural Networks with Ferroelectric Synapses for EEG Signal Processing

arXiv:2601.00020v3 Announce Type: replace-cross Abstract: Electroencephalography (EEG)-based brain-computer interfaces (BCIs) are strongly affected by non-stationary neural signals that vary across se

local-aiarxiv-cs-lg

ProgramBench: Can Language Models Rebuild Programs From Scratch?

arXiv:2605.03546v1 Announce Type: cross Abstract: Turning ideas into full software projects from scratch has become a popular use case for language models. Agents are being deployed to seed, maintain,

RAMoEA-QA: Hierarchical Specialization for Robust Respiratory Audio Question Answering

arXiv:2603.06542v2 Announce Type: replace-cross Abstract: Conversational generative AI is increasingly explored in healthcare, where models must integrate heterogeneous patient signals and support div

applicationsarxiv-cs-ai

Scalable Multi Agent Diffusion Policies for Coverage Control

arXiv:2509.17244v2 Announce Type: replace Abstract: We propose MADP, a novel diffusion-model-based approach for collaboration in decentralized robot swarms. MADP leverages diffusion models to generate

safetyarxiv-cs-ro

VCBench: Benchmarking LLMs in Venture Capital

arXiv:2509.14448v2 Announce Type: replace Abstract: Benchmarks such as SWE-bench and ARC-AGI demonstrate how shared datasets accelerate progress toward artificial general intelligence (AGI). We introd

When LLMs get significantly worse: A statistical approach to detect model degradations

arXiv:2602.10144v2 Announce Type: replace-cross Abstract: Minimizing the inference cost and latency of foundation models has become a crucial area of research. Optimization approaches include theoreti

A Framework for Exploring and Disentangling Intersectional Bias: A Case Study in Fetal Ultrasound

model-releasesarxiv-cs-cv

arXiv:2605.02942v1 Announce Type: cross Abstract: Bias in medical AI is often framed as a problem of representation. However, in image-based tasks such as fetal ultrasound, performance disparities can

A Neuro-Symbolic Framework for Accountability in Public-Sector AI

arXiv:2512.12109v3 Announce Type: replace-cross Abstract: Automated eligibility systems increasingly determine access to essential public benefits, but the explanations they generate often fail to ref

applicationsarxiv-cs-ai

A Unified Framework for Tabular Generative Modeling: Loss Functions, Benchmarks, and Improved Multi-objective Bayesian Optimization Approaches

arXiv:2405.16971v2 Announce Type: replace Abstract: Deep learning (DL) models require extensive data to achieve strong performance and generalization. Deep generative models (DGMs) offer a solution by

ABD: Default Exception Abduction in Finite First Order Worlds

arXiv:2602.18843v3 Announce Type: replace Abstract: We introduce ABD, a benchmark for default-exception abduction over finite first-order worlds. Given a background theory with an abnormality predicat

ADAPTS: Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms

arXiv:2605.03212v1 Announce Type: cross Abstract: Modeling latent clinical constructs from unconstrained clinical interactions is a unique challenge in affective computing. We present ADAPTS (Agentic

safetyarxiv-cs-cl

AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages

arXiv:2601.06395v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly multilingual, yet open models continue to underperform relative to proprietary systems, with the gap m

AfriVox-v2: A Domain-Verticalized Benchmark for In-the-Wild African Speech Recognition

arXiv:2605.03590v1 Announce Type: new Abstract: Recent large language models (LLMs) show strong speech recognition and translation capabilities for high-resource languages. However, African languages

Annotation Quality in Aspect-Based Sentiment Analysis: A Case Study Comparing Experts, Students, Crowdworkers, and Large Language Model

arXiv:2605.03624v1 Announce Type: new Abstract: Aspect-Based Sentiment Analysis (ABSA) enables fine-grained opinion analysis by identifying sentiments toward specific aspects or targets within a text.

Are LLMs More Skeptical of Entertainment News?

arXiv:2605.01727v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed stan

Audio-Visual Intelligence in Large Foundation Models

arXiv:2605.04045v1 Announce Type: new Abstract: Audio-Visual Intelligence (AVI) has emerged as a central frontier in artificial intelligence, bridging auditory and visual modalities to enable machines

safetyarxiv-cs-cv

Benchmarking Local Language Models for Social Robots using Edge Devices

arXiv:2605.03111v1 Announce Type: cross Abstract: Social-educational robots designed for socially interactive pedagogical support, such as the Robot Study Companion (RSC), rely on responsive, privacy-

local-aiarxiv-cs-cl

Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences

arXiv:2605.02584v1 Announce Type: cross Abstract: Agentic AI will be an essential enabling technology for designing future mobile communication systems, which could provide flexible and customized ser

Caliper-in-the-Loop: Black-Box Optimization for Hyperledger Fabric Performance Tuning

arXiv:2605.02690v1 Announce Type: cross Abstract: Hyperledger Fabric performance depends on many interacting configuration parameters, making manual tuning difficult. We study automated throughput tun

researcharxiv-cs-ai

Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology

arXiv:2605.03352v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated robust capabilities in recognizing everyday human activities, yet their potential for analyzi

researcharxiv-cs-cv

Can synthetic data reproduce real-world findings in epidemiology? A replication study using adversarial random forests

arXiv:2508.14936v3 Announce Type: replace-cross Abstract: Synthetic data holds substantial potential to address practical challenges in epidemiology due to restricted data access and privacy concerns.

Complexity Horizons of Compressed Models in Analog Circuit Analysis

arXiv:2605.02285v1 Announce Type: new Abstract: The deployment of Large Language Models (LLMs) for specialized engineering domains, such as circuit analysis, often faces a trade-off between reasoning

ContextCov: Deriving and Enforcing Executable Constraints from Agent Instruction Files

arXiv:2603.00822v2 Announce Type: replace-cross Abstract: As Large Language Model (LLM) agents increasingly execute complex, autonomous software engineering tasks, developers rely on natural language

Controllable and Verifiable Process Data Synthesis for Process Reward Models

arXiv:2605.02395v1 Announce Type: new Abstract: Process reward models (PRMs) rely on high-quality process supervision data, yet existing construction methods often provide limited control over error l

researcharxiv-cs-ai

Conventional Commit Classification using Large Language Models and Prompt Engineering

arXiv:2605.02033v1 Announce Type: cross Abstract: Conventional commits provide a structured format for writing commit messages, which improves readability, software maintenance, and enables automation

DocSync: Agentic Documentation Maintenance via Critic-Guided Reflexion

arXiv:2605.02163v1 Announce Type: cross Abstract: Software documentation frequently drifts from executable logic as codebases evolve, creating technical debt that degrades maintainability and causes d

Effective Performance Measurement: Challenges and Opportunities in KPI Extraction from Earnings Calls

arXiv:2605.03147v1 Announce Type: new Abstract: Earnings calls are a key source of financial information about public companies. However, extracting information from these calls is difficult. Unlike t

EO-Gym: A Multimodal, Interactive Environment for Earth Observation Agents

arXiv:2605.01250v1 Announce Type: new Abstract: Earth Observation (EO) analysis is inherently interactive: resolving uncertainty often requires expanding the region of interest, retrieving historical

EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triage

arXiv:2605.03998v1 Announce Type: new Abstract: Emergency department triage assigns patients an acuity score that determines treatment priority, and clinical evidence documents persistent gender dispa

From Code to Prediction: Fine-Tuning LLMs for Neural Network Performance Classification in NNGPT

model-releasesarxiv-cs-cv

arXiv:2605.03686v1 Announce Type: cross Abstract: Automated Machine Learning (AutoML) frameworks increasingly leverage Large Language Models (LLMs) for tasks such as hyperparameter optimization and ne

From Data Lifting to Continuous Risk Estimation: A Process-Aware Pipeline for Predictive Monitoring of Clinical Pathways

arXiv:2605.03895v1 Announce Type: new Abstract: This paper presents a reproducible and process-aware pipeline for predictive monitoring of clinical pathways. The approach integrates data lifting, temp

From prompting to evidence-based translation: A RAG+prompt system for Japanese-Chinese translation and its pedagogical potential

arXiv:2605.03387v1 Announce Type: new Abstract: Large language models perform well on high-resource pairs but are less reliable for Japanese-Chinese sentences containing noun-modifying clause construc

researcharxiv-cs-cl