Search: “benchmark”

SimGuide: Typed Multi-Context User Representations for Preference-Conditioned Agent Planning

arXiv:2608.24888v2 Announce Type: replace Abstract: Agents that act on a user's behalf must plan differently for different users, and increasingly do so from some structured representation of user con

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

SpIDER: Spatially Informed Dense Embedding Retrieval for Software Issue Localization

DGX agent

arXiv:2512.16956v3 Announce Type: replace-cross Abstract: Retrieving code functions, classes or files relevant to a user query, bug report or feature request from large codebases is a fundamental chal

model-releasesarxiv-cs-lg

1 Sept 2026

SwarmBench: Can Large Language Models Act as Agent Swarm Orchestrators?

arXiv:2608.30661v1 Announce Type: new Abstract: Large language model-based multi-agent systems are evolving from fixed interaction topologies toward dynamically orchestrated Agent Swarms. However, exi

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

arXiv:2603.22341v2 Announce Type: replace-cross Abstract: While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capt

The First Token Is a Clue: Verbalizing Multi-Token Concepts from the J-lens

arXiv:2608.31084v1 Announce Type: new Abstract: The Jacobian Lens (J-lens) is a recent tool for interpreting LLMs. It reads a hidden state as a ranked list of vocabulary tokens, leaving multi-token co

Agents

Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers

arXiv:2608.31067v1 Announce Type: new Abstract: Learning generalizable algorithmic computations remains a challenge for neural networks, as reflected in persistent failures on compositional and length

agentsarxiv-cs-lg

Update: llama.cpp for Radeon VII / MI50 / MI60 — +14% PP, +9% long-context fill vs upstream + adaptive Flash Attention

I posted a new gfx906 based llama.cpp fork a few days ago. One of the main points of critique was that i did not provide sufficient numbers for the gains to be achieved. -- TL;DR: After switching our

VibeJam: A User Study Platform for Web Development with Agents

arXiv:2608.29889v1 Announce Type: new Abstract: Programming with AI is increasingly agentic, users prompt LLMs to directly edit their code and review the changes, with adoption growing especially for

WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents

arXiv:2608.28062v2 Announce Type: replace Abstract: Multimodal search agents extend parametric knowledge with newly emerging and long-tail evidence from the open web. Yet many existing agentic search

When Similar Means Different: Evaluating LLMs on Arabic--Hebrew Cognates

arXiv:2606.13218v2 Announce Type: replace Abstract: Arabic and Hebrew, as closely related Semitic languages, share many words with similar surface forms, including true cognates, false friends, and mo

Whose Assessment of Distress? Community Perspectives and LLM Alignment on Well-Being Posts

arXiv:2608.29446v1 Announce Type: new Abstract: Judgments about psychological distress are socially situated: what counts as concerning hinges on community norms around emotional expression, vulnerabi

Safety

WildSEEK: Evaluating Language Models for Information-Seeking

arXiv:2608.30683v1 Announce Type: new Abstract: Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable in

safetyarxiv-cs-cl

Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores

arXiv:2608.31068v1 Announce Type: new Abstract: When a large language model fails a reasoning task, it is often assumed to lack the underlying capability. However, this conflates a genuine absence of

A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data

arXiv:2601.07576v2 Announce Type: replace-cross Abstract: Oral presentation skills are a critical component of higher education, yet comprehensive datasets capturing real-world student performance acr

Beyond Global Scalars: Synergizing Token-Level Statistics and Deep Semantics for Adversarial AIGC Text Detection

arXiv:2608.28009v1 Announce Type: new Abstract: The rapid evolution of large language models necessitates robust machine-generated text detection. Existing paradigms typically follow two isolated trac

CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents

arXiv:2608.28389v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) augments LLMs with external documents, but public or user-editable sources expose RAG systems to data poisoning:

Characterization of Request and Token Energy Costs for LLM Inference Workloads on GPU Platforms

arXiv:2608.28044v1 Announce Type: cross Abstract: Large language model (LLM) inference serving is priced by tokens, but GPU energy is consumed over inference windows. This accounting mismatch makes to

Dandelion: A Spherical Flower for Neural Simulation of Planetary Dynamics

arXiv:2608.27521v1 Announce Type: new Abstract: Many dynamical processes unfold on the sphere but the default scientific machine learning architectures are Euclidean. Applying these architectures on a

Deflation-PINNs: Learning Multiple Solutions for PDEs and Landau-de Gennes

arXiv:2603.27936v3 Announce Type: replace-cross Abstract: Nonlinear Partial Differential Equations (PDEs) are ubiquitous in mathematical physics and engineering. Although Physics-Informed Neural Netwo

Does anyone have real experience with Ornith-1.5-9B for coding

I'm very happy with Qwen-3.8-27B, I run it on my work machine and switched for major part of real coding tasks from cloud subscriptions to it. I use one dedicated headless RTX 3090, and get maybe 1000

Generalized Gibbs Ensemble Weighting for Forecast Combination

arXiv:2608.28116v1 Announce Type: new Abstract: Forecast combination is a reliable way to improve predictive performance when several forecasting models are available. Simple aggregation rules such as

Local Ai

Initialization Is Critical: Advancing Federated Short-Term Load Forecasting under Load Heterogeneity via Model Initialization

arXiv:2608.27791v1 Announce Type: new Abstract: Short-term load forecasting (STLF) provides essential information for numerous applications in modern power systems. However, accurate STLF often relies

local-aiarxiv-cs-lg

[new @activegraphai blog post] How Synthetic Players used ActiveGraph to verify 4,919 runs without new model calls https://activegraph.ai/bl…

model-releasesyohei-nakajima--x

[new @activegraphai blog post] How Synthetic Players used ActiveGraph to verify 4,919 runs without new model calls https://activegraph.ai/blog/synthetic-players-activegraph-replay For the increasing n

Physics-informed learning for the inverse problem in resonant ultrasound spectroscopy

arXiv:2608.27590v1 Announce Type: cross Abstract: Inferring elastic constants from resonant ultrasound spectra is a nonlinear and typically overdetermined inverse problem based on finite spectral data

Quantifying Affective Bias in Low-Resource Media: Large-Scale Emotion Profiling of Bengali Headlines

arXiv:2510.17252v2 Announce Type: replace Abstract: News media can influence readers not only through the events they report but also through the emotional tone used to present them. This issue is esp

Riverbank Erosion Analysis in Bangladesh Using Spatiotemporal Segmentation

arXiv:2510.17198v2 Announce Type: replace Abstract: Riverbank erosion is a serious environmental problem in Bangladesh, causing land loss, damage to infrastructure, and displacement of local communiti

Research

Roleplaying with Structure: Synthetic Therapist-Client Conversation Generation from Questionnaires

arXiv:2510.25384v2 Announce Type: replace Abstract: Large Language Models (LLMs) are promising tools for synthetic data generation in mental health. However, privacy policies and restrictions forced p

researcharxiv-cs-cl

Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning

arXiv:2604.06079v2 Announce Type: replace Abstract: Graphics Program Synthesis is pivotal for interpreting and editing visual data, effectively facilitating the reverse-engineering of static visuals i

Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection

arXiv:2509.24192v2 Announce Type: replace Abstract: Vision-language models (VLMs) have advanced multimodal perception, demonstrated by open-vocabulary object detection with simple language queries. St

The Effect of Emotional Context on Large Language Models' Endorsement of Premature Decisions: Comparing Emotional Vulnerability Across Six Commercial Models

arXiv:2608.27465v1 Announce Type: new Abstract: As large language models (LLMs) are increasingly used for everyday decision-making advice, whether a model shifts the direction of its advice according

Under-Mattress Temporal Sensing for Next-Day Agitation Risk Scoring in Dementia Wards

arXiv:2608.28152v1 Announce Type: cross Abstract: Agitation fluctuates over short time horizons in people living with dementia, yet continuous physiological information for anticipating next-day risk

What Will This Copper Look Like Later? Forecasting Surface Appearance and Rendering It as a PBR Material

arXiv:2608.28102v1 Announce Type: cross Abstract: Digital design requires predicting how a metal surface will look later in its oxidation; this paper presents such a pipeline for copper. Given a fixed

When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging

arXiv:2608.27518v1 Announce Type: new Abstract: Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by ca

1x32GB V100 vs 2x16GB V100 vs 5060ti 16GB for QWEN 3.8

Hi All, I am currently contemplating an upgrade from my 5060ti 16gb. I am getting ~40t/s with 130k context on Qwen 3.8 IQ3_S HF quant. I am running llama.cpp on linux. Objective is to increase context

Don't Sleep on EXL3 Quants

I'm running Muse Glimmer 30B EXL3-SC 3.00bpw H4, fully resident on my 12GB VRAM GPU at 100K context with Q8_O KV cache. It's a joy to use a dense 30B model at this size and still get 30 tok/s on a VRA

Local Ai

Qwen3.8-Flash-Next NVFP4 2xDGX Spark config: 50t/s decode, 2,900t/s prefill

After a couple of days of faffing around, here's my current config in case it helps anyone out. Some of this is also valuable for a single Spark. AI slop below for you to read and paste into your own

local-air-localllama

Ran Qwen3.8-Flash-Next (79 GB, 2-bit) at 350K ctx for 3.5 hours on a 128 GB M5 Max — speed vs context depth, 100 turns, one graph

Setup: MacBook Pro M5 Max, 128 GB unified, macOS 26.5.2 · llama.cpp b10686 (Metal, 12 threads, batch 2048, flash-attn, kv-unified, ngram-mod spec decode) · Qwen3.8-Flash-Next UD-Q2_K_XL (Unsloth), 78.

50% tg increase with offloading 'hot' experts to VRAM

I got a 50% performance boost (20 t/s -> 30 t/s) in llama.cpp for MoE models that don’t fit entirely in VRAM—in my case, Qwen 3.8 Flash Next. The idea is simple: instead of offloading entire layers to

29 Aug 2026

Banger paper from Apple. If you build MCP servers, this can help you turn your specification into an evaluation suite. (bookmark it) It's ac…

Banger paper from Apple. If you build MCP servers, this can help you turn your specification into an evaluation suite. (bookmark it) It's actually a really neat idea that's easy to implement. And it s

model-releasesdair-ai--x

29 Aug 2026

Qwen 3.8 27B at 50 tok/s with 100k Context on a 16GB GPU! (beellama.cpp)

I wanted to share my successful setup for running a Qwen 3.8 27B model with a massive context window on a consumer 16GB GPU (RTX 4070 Ti SUPER). The goal was to fit everything into VRAM without sacrif

29 Aug 2026

BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Models

arXiv:2608.27268v1 Announce Type: new Abstract: Although Large language models (LLMs) mediate access to knowledge and computational assistance, their capabilities should benefit vulnerable groups in t

COFM: Consistent Optimal Transport Flow Matching via Partially Input Convex Neural Networks

arXiv:2511.06042v2 Announce Type: replace Abstract: Optimal transport (OT) provides a principled framework for learning mappings between probability distributions, and has found broad applications in

Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms

arXiv:2608.27409v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves specific capabilities of large language models, but covering multiple capabilities often

Cross-Architecture Knowledge Distillation from a Vision Foundation Model to a Lightweight Visual State Space Model for Tea Leaf Disease Classification

arXiv:2608.26771v1 Announce Type: new Abstract: Automated tea leaf disease classification supports precision agriculture, yet deploying accurate models on edge devices remains challenging under tight

D2C-Routing: Dimension-to-Composition Evidence Routing for Mixed-Origin AI-Generated Text Detection

arXiv:2608.27380v1 Announce Type: new Abstract: AI-generated text detection is commonly framed as a binary document-level judgment about whether a text is human-written or machine-generated. This fram

Diffusion Policies for Short-Horizon Planning in Robot Crowd Navigation

arXiv:2608.27158v1 Announce Type: new Abstract: Robot crowd navigation requires safe and efficient decision-making under dense, dynamic, and multimodal human--robot interactions. Existing reinforcemen

Agents

Do LLMs Understand Personality? Rethinking Persona Fidelity Evaluation through Structured Behavioral Inference

arXiv:2608.26674v1 Announce Type: cross Abstract: As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an

agentsarxiv-cs-ai

DocTalkBN: A Novel Dataset of Expert Telemedicine Conversations in Bengali

arXiv:2608.27110v1 Announce Type: new Abstract: Reliable medical conversational AI requires authentic expert--patient interaction data, yet such datasets remain scarce, especially for low-resource lan