Search: “benchmark”

Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms

arXiv:2608.01004v1 Announce Type: new Abstract: Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to thei

4 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

AI9Stars released G9v3-39A5B

DGX agent

AI9Stars has released G9v3-39A5B an open weights language model designed to deliver even stronger reasoning capabilities than ai9stars/G9v3-3B with its 39B and 5 active experts. It is released under t

model-releasesr-localllama

3 Aug 2026

All frontier AI models (including GPT-5.6 Sol, Fable 5, and Kimi K3) consistently fail to implement solvers for basic nonlinear PDEs correct…

model-releasesgary-marcus--x

All frontier AI models (including GPT-5.6 Sol, Fable 5, and Kimi K3) consistently fail to implement solvers for basic nonlinear PDEs correctly, often introducing significant errors in numerical stabil

Can LLMs Really Understand Item Difficulty Levels? Implications for Automated Item Generation Using LLMs

arXiv:2607.28634v1 Announce Type: new Abstract: The estimation of item difficulty plays a key role in both formative assessment and large-scale high-stakes summative assessments. This study explores h

Detecting AI-Generated Videos with Spiking Neural Networks

arXiv:2605.05895v2 Announce Type: replace-cross Abstract: Modern AI-generated videos are photorealistic at the single-frame level, leaving inter-frame dynamics as the main remaining axis for detection

DiffAttack: Evasion Attacks Against Face Recognition via Latent Diffusion Models

arXiv:2607.28936v1 Announce Type: cross Abstract: Facial biometric identification relies on the distinctiveness of user attributes within a high-dimensional embedding space. However, the decision boun

Research

Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

arXiv:2511.02570v3 Announce Type: replace Abstract: Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate exper

researcharxiv-cs-lg

Evaluating Federated Pre-Training: On the Reliability of Downstream Fine-Tuning and Intrinsic Evaluation

arXiv:2607.28658v1 Announce Type: cross Abstract: Federated pre-training offers a way to train foundation models on private or distributed data without centralizing the underlying datasets. However, e

Federated Foundation Models Fine-Tuning with Heterogeneous Compressed Clients

arXiv:2607.29071v1 Announce Type: cross Abstract: Federated learning of foundation models faces a fundamental resource-asymmetry challenge: the institutions holding the most valuable domain-specific d

Frugal Bayesian Optimization: Scalable Surrogates for Data- and Resource-Limited Discovery

arXiv:2607.29225v1 Announce Type: new Abstract: Bayesian Optimization (BO) is widely adopted for data-efficient optimization in scientific and engineering applications, yet its computational cost is r

Is It Time for the Renaissance of Salient Object Detection in the Era of MLLMs?

arXiv:2607.29222v1 Announce Type: new Abstract: The zero-shot capabilities of multimodal large language models (MLLMs) are pushing salient object detection (SOD) beyond task-specific supervision. To d

Safety

JoVA: Unified Multimodal Learning for Joint Video-Audio Generation and Editing

arXiv:2512.13677v2 Announce Type: replace Abstract: In this paper, we present JoVA, a streamlined framework that unifies joint video-audio generation and editing. While existing methods often rely on

safetyarxiv-cs-cv

Learning Optimal Dynamic Matching via Graph Neural Networks

arXiv:2607.28925v1 Announce Type: new Abstract: Dynamic matching markets require decisions about whom to match and when: matching now yields value but removes participants who may create better future

Linear Proposal Operators and Stochastic Search Geometry in SOMA and Differential Evolution

arXiv:2607.29228v1 Announce Type: cross Abstract: Swarm and evolutionary algorithms are usually analyzed as complete procedural systems in which nonlinear selection, replacement, and adaptation obscur

M3MAD-Bench: Multi-Dimensional Evaluation of Multi-Agent Debate Across Domains and Modalities

arXiv:2601.02854v2 Announce Type: replace Abstract: As an agent-level reasoning and coordination paradigm, Multi-Agent Debate (MAD) orchestrates multiple agents through structured debate to improve an

MirrorCraft: Paired Evaluation under Hidden Rule Changes in Minecraft

arXiv:2607.29218v1 Announce Type: new Abstract: With the prosperity of the large language models (LLMs), it has become an interesting topic: how do LLM-based agents work in Minecraft? Unfortunately, m

On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness

arXiv:2607.29062v1 Announce Type: new Abstract: Model capabilities have improved in large part due to scaling chain of thought. This has been a promising development for AI safety--where models verbal

OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systems

arXiv:2607.28629v1 Announce Type: new Abstract: The rapid transition from reactive large language models (LLMs) to persistent, action-capable systems has exposed critical gaps in the architectural und

Retrieval-Driven Training-Free AI-Generated Video Attribution

arXiv:2607.28955v1 Announce Type: cross Abstract: AI-generated videos are becoming increasingly realistic and difficult to distinguish from authentic ones, which facilitates malicious misuse and poses

SILVA Networks as Structured Implicit Layers and Vector Attractors via Dynamic Interaction Fields

arXiv:2607.28989v1 Announce Type: new Abstract: Many learning problems require representations that reconcile direct input, nearby structure, and broader context. In implicit neural layers, these infl

Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?

arXiv:2607.28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect. We measure how often that treatment is

Research

When Model Priors Conflict with Visual Evidence: Mitigating Commonsense-Driven Hallucinations by Selective Prior Calibration

arXiv:2607.29240v1 Announce Type: cross Abstract: In vision--language models, commonsense-driven hallucination (CDH) occurs when a model's commonsense prior overrides clear visual evidence of an atypi

researcharxiv-cs-ai

Why It Hurts: Identifying the Drivers of Negative Thoughts in Emotional Support Conversations

arXiv:2607.28648v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for emotional support tasks, such as negative thought reframing. This task relies on modifying cogn

Best model <3B for multilingual understanding/ instruction following?

model-releasesr-localllama

I know qwen 3.5 4b is great but a bit too large and miniPCM5 1b is great for agentic use but not so great for multilingual natural language understanding. Google eXb variants are just too big in total

Deepseek v4 flash - 100-150 faster t/s in prefill/pp.

model-releasesr-localllama

You have two choices here (in order of pref): Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out) Use this vibed fork that works w

Hardware

I pushed Kimi K3 onto one CPU with 8 GB of RAM

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o

hardwarer-localllama

Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization - AI's narrative

model-releasesr-localllama

# Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization I used Deepseek-v4-Flash-0731 cloud API settig up vllm-moet to run deepseek-v4-flash with MTP locally on

3/ here's the part that makes it non-optional: the same agent that will do whatever it takes to solve a problem will also walk straight out …

model-releasesitamar-friedman--x

3/ here's the part that makes it non-optional: the same agent that will do whatever it takes to solve a problem will also walk straight out of a sandbox you thought was locked down. We watched exactly

1 Aug 2026

Trying to understand VRAM usage and find the sweet spot for Wan/SCAIL-2 (or other models) on a GPU

model-releasesr-stablediffusion

So upfront I'll admit that this is a ChatGPT summary of my chat with it about this idea i had, but this post wouldn't exist any other way, so... I’m trying to get a better understanding of how VRAM is

1 Aug 2026

What's currently the 'smartest' LLM to use on 8GB vram and 16 RAM and same thing for 8 VRAM and 64 RAM?

Been trying to find something that actually handles my workload well instead of just being 'fine.' Started on Qwen 2.5 7B, moved to Qwen 3 8B, and right now I'm using Nemotron 3 Ultra (the big 550B on

model-releasesr-ollama

1 Aug 2026

Safety

AI-assisted pre-review of open-source software submissions: an experience report from BOSC 2026

arXiv:2607.27228v1 Announce Type: new Abstract: Most conferences rely on peer-review of submissions, but as generative AI makes it easier than ever to prepare submission materials, some conferences ar

safetyarxiv-cs-cl

ARD-REFSM: Enhancing Reflection Symmetry Detection with Asymmetric Denoising and Rotation Equivariance

arXiv:2607.27927v1 Announce Type: new Abstract: Reflection symmetry detection remains challenging due to interference from asymmetric regions and arbitrary orientations of symmetric patterns. Asymmetr

AskChem: Claim-Centered Infrastructure for Chemistry Literature Synthesis

arXiv:2607.28618v1 Announce Type: new Abstract: Chemistry literature synthesis often requires assembling specific findings scattered across many publications, yet existing literature-search systems pr

Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding

arXiv:2607.28516v1 Announce Type: new Abstract: Long-video understanding commonly compresses videos into a small set of frames or visual tokens for answer generation. Existing compact pipelines focus

Applications

CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance

arXiv:2607.28292v1 Announce Type: new Abstract: Large Language Models (LLMs) deployed in dynamic financial environments face a critical challenge: maintaining factual accuracy as market conditions, re

applicationsarxiv-cs-cl

ChronoMem: Version Control and Semantic Rollback for Large Language Model Agent Memory

arXiv:2607.27773v1 Announce Type: new Abstract: LLM agents increasingly rely on long-term memory to support multi-session interaction and personalization. However, existing agent memory systems are de

Cross-Embodiment Transfer via Behavior-Aligned Representations

arXiv:2607.27549v1 Announce Type: cross Abstract: Recent progress in large-scale imitation learning for robot manipulation has been driven by leveraging datasets across a wide range of robot embodimen

EHGCN: Hierarchical Euclidean-Hyperbolic Fusion via Motion-Aware GCN for Hybrid Event Stream Perception

arXiv:2504.16616v4 Announce Type: replace Abstract: Event cameras, characterized by microsecond temporal resolution and very High Dynamic Range (HDR), emit high-speed event streams for perception task

EMBL AI Librarian: Life-Sciences Knowledge Layer for AI Agents

arXiv:2607.28229v1 Announce Type: new Abstract: The web is increasingly accessed by AI agents rather than humans. Every agent needs knowledge, especially in the life-sciences, where agentic pipelines

Evidence-Ledger Adjudication for Claim-Evidence Traceability

arXiv:2607.26512v1 Announce Type: new Abstract: AI agents can draft claims faster than authors can check whether the cited or retrieved evidence supports them. We study evidence-ledger adjudication: a

Hardware

FasTac: A Curved Multispectral Vision-Based Tactile Sensor for High-Speed High-Precision 3D Shape and Force Perception

arXiv:2607.28416v1 Announce Type: new Abstract: Curved tactile fingertips for dexterous manipulation must resolve fine contact geometry, distinguish normal and tangential loads, and capture transient

hardwarearxiv-cs-ro

Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

arXiv:2607.27275v1 Announce Type: new Abstract: Post-training quantization to 4-bit weights is widely reported to be nearly lossless. We test this claim for multi-turn, tool-calling agents, where it n

Safety

Functional Cache Grafting: Robust and Rapid Code-Policy Synthesis for Embodied Agents

arXiv:2606.13097v2 Announce Type: replace-cross Abstract: Code-writing large language models (CodeLLMs) generate executable code policies for embodied agents by translating natural language goals and

safetyarxiv-cs-ai

Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA

arXiv:2502.10497v2 Announce Type: replace Abstract: Recent advancements in Generative AI have significantly improved the efficiency and adaptability of natural language processing (NLP) systems, parti

IFHierBench: Hierarchical Instruction Following for Large Language Models

arXiv:2607.27912v1 Announce Type: cross Abstract: Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the ou

Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning

arXiv:2607.27610v1 Announce Type: new Abstract: Reinforcement learning (RL) finetuning significantly enhances the reasoning capabilities of large language models (LLMs), yet its effectiveness critical

Language Diversity: Evaluating Language Usage and AI Performance on African Languages in Digital Spaces

arXiv:2512.01557v3 Announce Type: replace Abstract: This study examines the digital representation of African languages and the challenges this presents for current language detection tools. We evalua

Research

LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models

arXiv:2607.28077v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rol

researcharxiv-cs-cl