Search: “benchmark”

ReguSim: Evaluating LLM Agent Rule Grounding in Financial Compliance

arXiv:2608.19974v1 Announce Type: new Abstract: LLM agents in financial markets may cite rules yet still submit orders that violate executable constraints or misread surveillance evidence. We introduc

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation

DGX agent

arXiv:2607.14497v2 Announce Type: replace Abstract: Egocentric Visual Question Answering (VQA) has attracted widespread attention as an important task for enabling Multimodal Large Language Models (ML

model-releasesarxiv-cs-cv

21 Aug 2026

Robust Cross-Modal Foundation Model Perception for Underwater Robots under Degraded Visual Conditions

arXiv:2608.19710v1 Announce Type: cross Abstract: Reliable underwater robotic perception remains difficult because optical imagery degrades under turbidity, wavelength-dependent attenuation, low illum

Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale

arXiv:2608.19625v1 Announce Type: new Abstract: Scientific data are increasingly used by AI agents, yet existing dataset representations provide limited support for autonomous discovery, interpretatio

SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis

arXiv:2601.18305v2 Announce Type: replace Abstract: Despite numerous Graphical User Interface (GUI) agents claiming to automate user interaction tasks, to date, few achieve satisfactory interaction ca

Time-Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life

arXiv:2608.19218v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are increasingly being explored for domain-specific maintenance and prognostics tasks, raising the

UrbanVGGT: Scalable Sidewalk Width Estimation from Street View Images

arXiv:2603.22531v2 Announce Type: replace Abstract: Sidewalk width is an important indicator of pedestrian accessibility, comfort, and network quality, yet large-scale width data remain scarce in most

A Unifying Relational Perspective on Expressive Lottery Tickets

arXiv:2608.18819v1 Announce Type: new Abstract: Graph neural networks (GNNs) are widely used, but how parameter sparsity affects the expressivity of relational (RGNNs) and temporal (TGNNs) variants is

Local Ai

Aurora-80K releases! A modern tiny language model.

I'm introducing Aurora-80K, a small language model with exactly 80 thousand parameters. It uses a factorized 4,096-token vocabulary despite having only 80K parameters. The benchmarks: Wikitext-2 BPB:

local-air-localllama

Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution

model-releasesarxiv-cs-cl

arXiv:2608.18575v1 Announce Type: new Abstract: Large language model (LLM)-based multi-agent systems (MAS) often exhibit complex failure modes, which frequently cause agents to produce incorrect outco

BREAKING: Grok 4.6 just tied for #1 on the Artificial Analysis Agentic Index.⚡🤖 • Grok 4.6 (High) — 59 • Claude Opus 5 (Max) — 59 Grok 4.6 …

model-releaseselon-musk--x

BREAKING: Grok 4.6 just tied for #1 on the Artificial Analysis Agentic Index.⚡🤖 • Grok 4.6 (High) — 59 • Claude Opus 5 (Max) — 59 Grok 4.6 is now at the top of the benchmark, ahead of other leading mo

Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference

arXiv:2608.18591v1 Announce Type: new Abstract: Uniformly allocating inference reasoning budgets to LLMs is expensive and prone to over-thinking penalties; especially in document tasks where visual la

[Draft - Open PR] AVX2: Speed up large batch size prompt processing of IQ models by bartowski1182 · Pull Request #27402 · ggml-org/llama.cpp

model-releasesr-localllama

IQ quants are particularly slow on CPU at large batch sizes (what you'd see for imatrix and perplexity) Benchmark numbers I ran PPL against master and this PR to get speed and numbers on --chunks 50 f

Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application

arXiv:2608.18289v1 Announce Type: new Abstract: The extraction of structured information from unstructured documents represents a critical component of digital transformations in all sectors. While pr

FraudBench: Stress-Testing Policy-Grounded Banking Agents Against Adaptive Fraud

arXiv:2608.18136v1 Announce Type: new Abstract: Conversational agents now act for end users through tools while holding access to customer databases and internal policy documents that a caller can rea

Jailbreaking in the Haystack

arXiv:2511.04707v2 Announce Type: replace-cross Abstract: Recent advances in long-context language models (LMs) have enabled million-token inputs, expanding their capabilities across complex tasks lik

Agents

Looped Language Models Improve Compositional Tool Calling

arXiv:2608.18171v1 Announce Type: new Abstract: Looped language models have shown promising results on reasoning benchmarks, yet their potential for agentic tool use remains largely unexplored. We stu

agentsarxiv-cs-ai

Low-Power, Neuromorphic, Acoustic Anomaly Detection for Persistent Machine Monitoring

arXiv:2608.18341v1 Announce Type: cross Abstract: Persistent acoustic monitoring can detect machine faults without physical contact, but always-on inference is constrained by power, latency, and deplo

Mise-en-Scene: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation

arXiv:2608.19000v1 Announce Type: new Abstract: Automating graphic design synthesis from user-provided elements requires both a coherent overall composition and the exact preservation of each asset. E

MissDiag: Diagnostic Evaluation of Incomplete-Knowledge Robustness in KGQA and KG-RAG

model-releasesarxiv-cs-cl

arXiv:2608.18489v1 Announce Type: new Abstract: Knowledge graph question answering (KGQA) and knowledge-graph-based retrieval-augmented generation (KG-RAG) aim to ground answers in explicit graph evid

Applications

Position: AI Leaderboards Are Underserving the Global South: A Case Study from India

arXiv:2608.18117v1 Announce Type: new Abstract: This position paper argues that AI leaderboards are structurally ill-suited to serving the Global South because they lack independent governance, confli

applicationsarxiv-cs-ai

Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets

arXiv:2608.19147v1 Announce Type: cross Abstract: Modern Intel AI PCs ship capable integrated GPUs and NPUs with 16+ GB of unified memory, and they spend considerable time idle. That is not enough mem

Qwen3.8-27B scored 29/30 on AIME 2026 with FP8 + xhigh reasoning — BF16 vs FP8 results

model-releasesr-localllama

I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium

Agents

Qwen3.8-27B took a serious hit to knowledge vs 3.6

Like many of you I've spent the last few days throwing Qwen3.8-27B against all of my usual use-cases and personal tasks/harnesses and workflows. It's great, phenomenal sometimes, but that's not what t

agentsr-localllama

SPADE: Self-Play in Adaptive Synthetic Executable Environments

arXiv:2608.19197v1 Announce Type: cross Abstract: Continuous self-improvement requires an ever-expanding pool of self-generated, diverse, adaptive goals. For language agents, existing training environ

StocksTalk: A Voice-Enabled Conversational Agent for Structured Query Generation over Web Data

arXiv:2608.18105v1 Announce Type: cross Abstract: StocksTalk is a voice-enabled conversational system for transforming spoken financial screening requests into executable and validated structured quer

The Deontic Gap: Large Language Models and the Modal Language of Obligation

arXiv:2608.18144v1 Announce Type: cross Abstract: Modal auxiliaries such as must, should, and have to mark necessity and obligation within the contexts of speaker authority and interpersonal stance. W

The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

arXiv:2608.18300v1 Announce Type: new Abstract: LLM-as-a-Judge, which leverages a large language model to evaluate natural language generated by another AI application or model, has become a standard,

Training Chemical Plausibility-Aware Large Language Models for Single-Step Retrosynthesis

arXiv:2608.18940v1 Announce Type: cross Abstract: Single-step retrosynthesis is a central component of computer-aided synthesis planning, yet its intrinsically one-to-many nature is poorly captured by

TrojanGYM: A Detector-in-the-Loop LLM for Adaptive RTL Hardware Trojan Insertion

arXiv:2601.17178v3 Announce Type: replace-cross Abstract: Hardware Trojans (HTs) remain a critical threat because learning-based detectors often overfit to narrow trigger/payload patterns and small, s

VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

arXiv:2608.18607v1 Announce Type: new Abstract: Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combi

Research

Which Negatives Matter? Ask Your Text Encoder: Adaptive Similarity Margins for Dense-Caption Retrieval

arXiv:2608.18521v1 Announce Type: new Abstract: Dense-caption retrieval has recently been improved by introducing segmentation, edge maps, LLM-filtered captions, and cross-modal modules into contrasti

researcharxiv-cs-ai

ArborMem: Navigating Interaction States with Memory Forests

model-releasesarxiv-cs-cl

arXiv:2608.17534v1 Announce Type: new Abstract: Large language models increasingly serve as persistent conversational assistants, requiring memory that preserves relevant experience and maintains cont

DMT-Dens: Density-preserving manifold visualization for biological data

arXiv:2608.17571v1 Announce Type: cross Abstract: Motivation: Low-dimensional embeddings are widely used to explore cell-state heterogeneity in single-cell and other high-dimensional biological data.

OOD Detection for EEG-based Machine Learning in High-Risk Environments

arXiv:2608.17620v1 Announce Type: new Abstract: Machine learning models for electroencephalography (EEG) analysis show great promise across a wide range of applications, but their deployment in high-r

Ornith-1.5 9B might not be bad at all

model-releasesr-localllama

(my own experience) I have a mid-range gaming laptop from 2024 with 4GB VRAM and 16GB RAM. There is nothing that I can run other than small models at humble t/s. I mostly use the models to generate sc

Pathology Transport: Optimal-Transport Explanations for Clinical Data, and When Their Heatmaps (Fail to) Localize Disease

arXiv:2608.17370v1 Announce Type: new Abstract: Generative models promise a route to explainable clinical AI: rather than probe a classifier, model the distributions of healthy and diseased patients a

SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version

arXiv:2608.17164v1 Announce Type: new Abstract: Textual context such as news, reports, and logs can provide valuable signals for time series forecasting, especially when future dynamics are driven by

Safety

Bias-Corrected Ceilings of Emotion Predictability from Human Label Variation Based on Instance-Level Fano Bounds

arXiv:2608.15619v1 Announce Type: new Abstract: Emotion recognition from text keeps improving on benchmarks, yet whether an accuracy ceiling has been reached is seldom asked with discipline. Our aim i

safetyarxiv-cs-ai

Research

CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?

arXiv:2608.16829v1 Announce Type: cross Abstract: Video world models approximate the stochastic distribution of physical outcomes through generative sampling, but existing benchmarks score individual

researcharxiv-cs-ai

Certifying Compressed Language Models: An Audit and a Statistical Toolkit

arXiv:2608.15046v1 Announce Type: new Abstract: A fraction of a point of benchmark accuracy is the usual evidence that a compressed model is equivalent to its original. That quantity is least informat

Deep Learning for BioImaging: What Are We Really Learning?

arXiv:2603.13377v2 Announce Type: replace Abstract: Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscop

DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

model-releasestogether-ai-blog

In the DeepSWE benchmark a two‑stage approach of running DeepSeek V4 Pro 0813 first and falling back to GPT‑5.6 Sol when the tests fail achieves 83 % success at an average cost of 3.35 per task, compa

Evaluating Agentic Code Repair Capabilities in Distributed Systems

arXiv:2608.14863v1 Announce Type: cross Abstract: LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified.

Local Ai

Is Ling 3 tiny underrated for its size?

I was checking out benchmarks of this model and apparantly better than Qwen3.5 9b reasoning across the bench on artificial analysis. I have used the 9b model for variety of stuff and it has been amazi

local-air-localllama

Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair

arXiv:2608.15579v1 Announce Type: cross Abstract: Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long

Large Language Models Show Metacognitive Sensitivity in Medical Reasoning

arXiv:2608.14552v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated and used in medicine, but clinical usefulness depends on answer accuracy and whether confidence

Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization

arXiv:2608.16072v1 Announce Type: cross Abstract: Reinforcement learning (RL) with group-relative advantages has become the de facto standard for post-training language model reasoners. However, when