Search: “benchmark”

From Visual Widgets to UI Code: Efficient Tool-Grounded Generation

arXiv:2608.12611v1 Announce Type: new Abstract: Existing screenshot-to-code systems face a trade-off between flexibility and controllability. Direct multimodal generation can hallucinate visible detai

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition

DGX agent

arXiv:2608.12683v1 Announce Type: cross Abstract: Embodied agents must often identify and interact with objects based on their function rather than their identity, requiring them to actively acquire o

model-releasesarxiv-cs-cv

14 Aug 2026

Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval

arXiv:2608.12843v1 Announce Type: cross Abstract: Text-based person anomaly retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural language desc

MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning

arXiv:2608.12724v1 Announce Type: new Abstract: Few-shot in-context learning (ICL) with multi-modal large language models (MLLMs) enables task adaptation without parameter updates, but its performance

Z.ai debuts GLM-5.3 with long-horizon coding, cybersecurity upgrades

model-releasessiliconangle

Chinese artificial intelligence developer Z.ai Co. today debuted GLM-5.3, an open-source large language model that set records across several popular benchmarks. The LLM is based on an algorithm calle

Safety

Accuracy and Order Sensitivity Diverge Under Label-Free Strategies

arXiv:2608.11947v1 Announce Type: cross Abstract: Multiple-choice benchmarks are widely used to evaluate large language models, but MCQ scores conflate knowledge with sensitivity to option order, whic

safetyarxiv-cs-ai

BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents

arXiv:2511.20597v2 Announce Type: replace-cross Abstract: The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web applica

Cross-Corpus Evaluation of Generalizable Vulnerability Detection in IoT Firmware

arXiv:2608.11492v1 Announce Type: cross Abstract: IoT firmware vulnerability detection remains challenging due to heterogeneous firmware ecosystems, resource-constrained platforms, and limitations in

Gemma 4 12B Q3: +8.55% Coding Performance From Tensor-Level Quantization Allocation

Ive been experimenting with task-aware GGUF quants for months, taking inspiration from TASA and TAQO but pushing the allocation lower to to the tensor level. The basic idea is to generate a custom ima

Research

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608.11922v1 Announce Type: new Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidat

researcharxiv-cs-cl

OEIS Open: How many conjectures can language models turn into theorems?

arXiv:2608.11941v1 Announce Type: new Abstract: We construct OEIS Open, a benchmark based on 492 open mathematical conjectures from the OEIS, formalized in Lean by Tsoukalas et al. Whereas these conje

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505.03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-triv

Research

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

arXiv:2608.11150v1 Announce Type: new Abstract: While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle

researcharxiv-cs-cv

DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains

arXiv:2608.11154v1 Announce Type: new Abstract: Detecting or attributing a supply-chain disruption is not the same as selecting the intervention that maximizes recoverable net value. We present Critic

FADE: From Passive Verification to Active Discovery in Counterfactual Video Understanding

arXiv:2608.10764v1 Announce Type: new Abstract: Counterfactual video understanding evaluates whether models grasp physical and commonsense regularities. However, existing multiple-choice question (MCQ

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

arXiv:2509.13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. Whi

HNDiff: Haze-Noise Diffusion for Image Dehazing

arXiv:2608.10995v1 Announce Type: new Abstract: Existing diffusion-based methods have recently made significant progress in image dehazing. However, they typically neglect the physics of haze formatio

Interesting research suggests caution in determining which AI company is winning by looking at any one source.. OpenRouter seems to show ope…

model-releasesethan-mollick--x

Interesting research suggests caution in determining which AI company is winning by looking at any one source.. OpenRouter seems to show open weights winning over time, but work submitted to Pangram i

Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility

arXiv:2608.10315v1 Announce Type: cross Abstract: Large language models (LLMs) are powerful black-box systems, making it difficult to discern whether their answers reflect stable internal beliefs or s

Mapping and Measuring the Behavioral Evolution of Large Language Models

model-releasesarxiv-cs-cl

arXiv:2608.11027v1 Announce Type: cross Abstract: Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across genera

No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding

model-releasesarxiv-cs-cl

arXiv:2503.05061v3 Announce Type: replace Abstract: Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expands, particularly in high-stakes domains such as bus

Nutrition Data Infrastructure for the AI Era: Operationalizing FAIR for Agent-Mediated Research

arXiv:2608.10363v1 Announce Type: new Abstract: AI agents can accelerate nutrition research, but their analyses inherit the identity, semantic, and release ambiguities of the underlying data. We prese

Optimal Stopping of Self-Refining Foundation Models

arXiv:2608.10729v1 Announce Type: cross Abstract: Foundation models can improve their outputs through a self-refinement process driven by external feedback. In this process, the model is embedded in a

Safety

Physics-Informed Machine Learning in Prognostics and Health Management: A Systematic Literature Review

arXiv:2608.10047v1 Announce Type: cross Abstract: In modern industry, keeping complex systems reliable, safe, and efficient hinges on Prognostics and Health Management (PHM). Machine Learning (ML) has

safetyarxiv-cs-ai

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608.10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interact

Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

arXiv:2608.11191v1 Announce Type: cross Abstract: GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ab

Agents

You chose the best model. Why is your agent still failing?

Public benchmarks can show how a model performs in general. Production reliability depends on the context and harness around it, which only your team can evaluate against its own data, workflows, and

agentsarize-ai

10 year garbage card for local llms

Hello everyone! I like dumb things. I like working with weak computers and microcontrollers. I like the simplicity and low electricity usage. Simply put, the efficiency of a 'dumb' PC. The first tim

AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images

arXiv:2608.08874v1 Announce Type: new Abstract: Spoken language offers a natural, hands-free interface for specifying an arbitrary target in dense remote-sensing imagery, yet existing referring remote

Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction

arXiv:2608.08459v1 Announce Type: cross Abstract: Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domain

Building Agent Skills and testing them is hard, but it doesn't have to be. Listen to Arjun Patel demo Cultivar, an open source tool develope…

model-releasespinecone--x

Building Agent Skills and testing them is hard, but it doesn't have to be. Listen to Arjun Patel demo Cultivar, an open source tool developed at Pinecone to help benchmark agent skills in sandboxes. T

Control-Diverse Reinforcement Fine-Tuning: Decoupling the Shared Control Bottleneck of RL Post-Training

arXiv:2608.08224v1 Announce Type: new Abstract: Reinforcement learning post-training unlocks complex reasoning in LLMs. Yet benchmark scores reveal only whether a model improved, not what changed insi

DarwinX: Evolving Agent Harnesses Through Natural Selection

arXiv:2608.07545v1 Announce Type: cross Abstract: An LLM agent's capability depends not only on model weights but on its harness: prompts, tools, skills, and control flow. Self-improvement loops alrea

DeepSeek-V4-Flash-0731 (284B MoE) at 75 tok/s on 2× DGX Spark — full recipe, 11 gotchas, reboot-proof cluster, Codex CLI integration

Spent two nights getting deepseek-ai/DeepSeek-V4-Flash-0731 (284B MoE, 13B active, native FP4/FP8, 1M context) running production-grade on two DGX Sparks connected by one QSFP DAC cable. Everything —

Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Families

arXiv:2608.08029v1 Announce Type: cross Abstract: Khatri et al. (2026) [DOI: 10.1109/DSN-W70714.2026.00027] show that lightweight MLP probes on final-layer activations of a single 8B model (LLaMA-3.1-

Safety

Evaluation of Motivational Interviewing Counsellors with Task-Aware Multi-Stage LLM-Based Simulated Clients

arXiv:2608.07499v1 Announce Type: cross Abstract: The development and benchmarking of Large Language Model (LLM)-based Motivational Interviewing (MI) counsellors now often rely on LLM-based simulated

safetyarxiv-cs-ai

Tutorials

Experience-Sensitive Game Learning: A Behavioral Study of Humans and Language Agents

arXiv:2608.07490v1 Announce Type: cross Abstract: Large language model agents are increasingly evaluated through games, but most benchmarks emphasize final outcomes rather than how players learn from

tutorialsarxiv-cs-ai

Floating-Point Neural Network Verification at the Software Level

arXiv:2510.23389v2 Announce Type: replace-cross Abstract: The behaviour of neural network components must be proven correct before deployment in safety-critical systems. Unfortunately, existing neural

ForestBench: A Unified Graph Framework for Evaluating Multi-Agent Collaboration

arXiv:2608.08605v1 Announce Type: new Abstract: Multi-agent systems (MAS) built on Large Language Models (LLMs) are proliferating rapidly, but their heterogeneous execution traces provide no common ba

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

arXiv:2608.09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet exi

Research

MDB-Link: Hierarchical Schema Linking for Multi-Database Text-to-SQL

arXiv:2608.09588v1 Announce Type: new Abstract: Traditional Text-to-SQL research and benchmarks assume a known target database, overlooking settings in which a query must be routed within a large, het

researcharxiv-cs-cl

Applications

Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses

arXiv:2608.07946v1 Announce Type: cross Abstract: Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean. Production warehouses are the inverse: cryptic identifiers,

applicationsarxiv-cs-ai

Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic

arXiv:2601.22510v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often achieve strong benchmark accuracy yet remain brittle under small distribution shifts. While recent mechanis

SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment

arXiv:2608.07639v1 Announce Type: cross Abstract: Agent Skills provide reusable capabilities to LLM agents. Agent Skill inconsistencies can expose undisclosed dangerous behavior or cause wrong Skill s

SkillReason: Reasoning-Enhanced Agent Skill Retrieval for Implicit User Requests

arXiv:2608.08640v1 Announce Type: new Abstract: Large language model agents increasingly rely on reusable skills to extend their capabilities beyond parametric knowl- edge. However, retrieving the app

TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis?

arXiv:2608.07899v1 Announce Type: new Abstract: Agent systems increasingly expose execution traces, yet telemetry that reveals a failure may still be inadequate for identifying where that failure orig

Agents

We built the Agentic World Cup - LLMs that compete in 1v1 Soccer. [P]

Hey everyone - we've been building something particularly relevant to ML at large - The Agentic World Cup - a platform where Agents compete in sports. As you know, today's Agents can code, do math, an

agentsr-machinelearning

Achievable 253 t/s - unsloth/Muse Glimmer 30B UD-Q5_K_M on a 5090

Benchmarked Muse Glimmer 30B on my RTX 5090 (32GB), 262k context, UD-Q5_K_M + dflash-kquant + mmproj. Workload Stock master + DFlash ngram-simple PR #26842 + DFlash Code patch 78 t/s 57 t/s 220-253 t/