Search: “benchmark”

Can Agentic Trading Systems Pay for Their Own Intelligence?

arXiv:2607.10286v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs tha

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators

DGX agent

arXiv:2608.30110v1 Announce Type: cross Abstract: Nowcasting headline macroeconomic indicators, i.e., estimating an indicator's value for the current reference period before its official release, is c

model-releasesarxiv-cs-ai

1 Sept 2026

Claude Fable 5.1 made me a really nice animated pelican

model-releasessimon-willison

Today is Claude Fable (and Mythos) 5.1 day. Anthropic say that Fable 5.1 'sets a new standard for coding, knowledge work, and long-running problem-solving tasks'. Their announcement spends a notable a

Confidence-Aware Ensemble and Long-Word Refinement for Artistic Text Recognition

arXiv:2608.29970v1 Announce Type: new Abstract: Artistic Text Recognition (ATR) remains challenging because word images often combine decorative fonts, curved layouts, object-like characters, clutter,

Defending Wearable VLMs Against Private Attribute Inference

arXiv:2608.28691v1 Announce Type: cross Abstract: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture: a user asks a task-driven question about the surroundi

En-ViMedNER: An English-Vietnamese Parallel Biomedical Corpus with UMLS Semantic Type Annotations

arXiv:2608.29890v1 Announce Type: new Abstract: Biomedical Named Entity Recognition (NER) is fundamental to healthcare AI applications, including clinical decision support and medical information extr

Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models

arXiv:2608.30884v1 Announce Type: cross Abstract: While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Exist

EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants

arXiv:2608.29387v1 Announce Type: new Abstract: Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests e

EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents

arXiv:2608.30429v1 Announce Type: new Abstract: LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution.

GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures

arXiv:2608.30858v1 Announce Type: cross Abstract: Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions.

Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam

arXiv:2608.29959v1 Announce Type: new Abstract: Language is humanity's most consequential technology, yet for over a billion speakers across India's twenty-two constitutionally recognised languages, i

Gurukul AI: An Interactive AI-Driven Educational Platform for Indian Education System

arXiv:2608.28611v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) like ChatGPT and LLaMA have transformed AI-driven education, but these systems are predominantly train

MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents

arXiv:2608.31022v1 Announce Type: new Abstract: AI agents in partially observable environments need to coordinate active sensing with working memory to maintain an evolving perceptual state. However,

SHADOWBENCH: Toward Reliable Automatic Evaluation of Semantic Alignment in Autoformalization

arXiv:2608.29270v1 Announce Type: cross Abstract: Autoformalization translates informal mathematical theorems into code for proof assistants such as Lean. A central challenge is that current evaluatio

SinLlama -- A Large Language Model for Sinhala

arXiv:2508.09115v5 Announce Type: replace Abstract: Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing mul

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation

arXiv:2601.16753v2 Announce Type: replace-cross Abstract: Longitudinal information in radiology reports refers to the sequential tracking of findings across multiple examinations over time, which is c

TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models

arXiv:2608.29999v1 Announce Type: cross Abstract: Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify. De

The Price of Intelligence: A Quality-Adjusted Price Index for AI Services

model-releasesarxiv-cs-lg

arXiv:2608.29843v1 Announce Type: cross Abstract: Posted prices for AI inference have fallen steadily since 2024, yet the measured speed of that fall depends almost entirely on the method of measureme

VocalAffectBench: Evaluating Vocal Emotion Recognition in AI Audio Models

arXiv:2608.28932v1 Announce Type: new Abstract: Voice products increasingly need affective cues that are present in speech but absent from transcripts. We introduce VocalAffectBench, a public, test-on

Safety

When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents

arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active st

safetyarxiv-cs-cl

WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

arXiv:2607.18084v2 Announce Type: replace Abstract: Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear predictio

CF-YOLO: Context-Aware Feature Refinement for Camouflaged Industrial Micro-Defect Detection

arXiv:2608.28070v1 Announce Type: new Abstract: Automated detection of surface micro-defects on industrial components, such as copper tubes, is critically important for quality assurance but remains c

Compact Rollback MTP: a MTP version for QWEN models for those with little vRAM

model-releasesr-localllama

I've made a modification of llama.cpp MTP for people that want to run models like QWEN 27B on 16GB and similar setup, the focus is reducing the memory cost of MTP allowing more speed for less ctx cost

Comparing Classical and Quantum Machine Learning for Regression in High Energy Physics Collision Data

model-releasesarxiv-cs-lg

arXiv:2608.28084v1 Announce Type: new Abstract: The classification and regression of particle collision events constitute a persistent computational challenge in experimental high energy physics, wher

Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging

arXiv:2608.28341v1 Announce Type: new Abstract: Precise dense correspondence is a fundamental prerequisite for multimodal spectral imaging systems that fuse disparate wavelength ranges for subsequent

Improved off-policy training of diffusion samplers

model-releasesarxiv-cs-lg

arXiv:2402.05098v5 Announce Type: replace Abstract: We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark s

NumBench: Diagnosing Counting Failures in Text-to-Image Models

arXiv:2608.28206v1 Announce Type: new Abstract: Text-to-image (T2I) models often generate the wrong number of objects, yet existing benchmarks are too small or weakly controlled to explain why. We int

Post-Training VLMs for Video Mistake Detection

arXiv:2608.28406v1 Announce Type: new Abstract: Human mistakes are inevitable when following instructions, yet they can lead to severe consequences. As such, there has been an increased interest in de

SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correction

arXiv:2511.14684v2 Announce Type: replace Abstract: Large language models (LLMs) often make reasoning errors when solving mathematical problems, and how to automatically detect and correct these error

an underrated part of glm-5.3 is its low hallucination rate claude fable 5 is over 2x higher, while gpt-5.6 luna is over 3x higher

model-releasestogether-ai--x

GLM‑5.3 demonstrates a notably lower hallucination rate compared with other contemporary large language models. According to a report by Together AI at TogetherCompute (published August 30 2026), Clau

30 Aug 2026

I fine-tuned a 0.8B local model for dictation cleanup. It matched a hosted frontier model on this narrow task

model-releasesr-localllama

I built SpeakoFlow Mini, an Apache-2.0 fine-tune of Qwen3.5-0.8B for dictation cleanup. It is not a chat model or a general rewriter. It takes speech-to-text output, applies corrections the speaker ac

30 Aug 2026

Try GLM 5.3!

Users are encouraged to try the newly released GLM 5.3 through the `ollama` platform, as announced by @ZixuanLi_ on Twitter. A new benchmark release—Terminal‑Bench 4.0—was also dropped, highlighting t

model-releasesollama--x

29 Aug 2026

4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation

arXiv:2608.26947v1 Announce Type: cross Abstract: Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large

Accelerating Scientific Research with Gemini in the Real-World

arXiv:2608.26701v1 Announce Type: new Abstract: We present an extension and comprehensive real-world validation of Co-Scientist, a Gemini-based multi-agent system designed to accelerate end-to-end sci

Are Large Language Models Effective Knowledge Graph Constructors?

arXiv:2510.11297v2 Announce Type: replace Abstract: Knowledge graphs (KGs) are widely used in knowledge-intensive applications, yet it remains unclear how effectively current large language models (LL

BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click

arXiv:2608.26867v1 Announce Type: new Abstract: Large language model agents reason, call tools, and act autonomously over many steps, but their agentic skills-correctly sequencing tools, planning unde

Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations

arXiv:2608.27066v1 Announce Type: cross Abstract: Privacy-Enhancing Technologies (PETs) in computer vision often rely on noise or image perturbations to protect visual data while securely processing i

Beyond Factual QA: Mentorship-Oriented Question Answering over Long-Form Multilingual Content

arXiv:2601.17173v2 Announce Type: replace-cross Abstract: Question answering systems are typically evaluated on factual correctness, yet many real-world applications-such as education and career guida

CIFQA: A Deterministic Tool-Grounded Multi-Agent LLM Framework for Financial Query Answering

arXiv:2608.26114v1 Announce Type: new Abstract: Calculation-intensive financial question answering requires exact reasoning over structured rates, temporal conditions, numerical formulas, and rule-bas

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

arXiv:2608.26757v1 Announce Type: new Abstract: Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities

Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?

arXiv:2607.15439v2 Announce Type: replace Abstract: Our previous ARC-AGI-3 agent bundled executable world modeling, prompted simplification, and exact replay verification, leaving their individual con

Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI

arXiv:2608.26442v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, ma

ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elements

arXiv:2608.26118v1 Announce Type: new Abstract: Existing long-form factuality evaluation relies on the decompose-retrieve-verify pipeline. However, the pipeline suffers from noise from claim decomposi

Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds

arXiv:2608.26743v1 Announce Type: new Abstract: Enterprises fine-tune language models on proprietary data that may later require removal due to privacy, contractual, or compliance obligations. Selecti

Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Prevention

arXiv:2608.26168v1 Announce Type: cross Abstract: The lifecycle of hallucination in LLMs is a concept that enables building solid frameworks on the control and reliability of LLMs in high-stakes envir

How Unlikely Is 'Unlikely'? Assessing Verbal Probability Perception Across Large Language Models

arXiv:2608.26327v1 Announce Type: cross Abstract: Large language models increasingly produce and interpret verbal probability expressions, yet whether these expressions carry consistent meaning across

Invocation-Level Reliability of Tool-Using Agents

arXiv:2608.26189v1 Announce Type: new Abstract: Tool-using agents fail two ways: choosing the wrong tool, or forming wrong arguments, and an early failure of either kind can silently corrupt everythin

Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives

arXiv:2608.26372v1 Announce Type: cross Abstract: Large language models are increasingly deployed as autonomous agents serving users on behalf of companies, placing them in settings where user and dep