Search: “benchmark”

arXiv:2609.07884v1 Announce Type: new Abstract: Recent work, such as Vision Banana, shows that lightweight instruction tuning can enable an image generator to achieve state-of-the-art performance acro

arXiv:2609.07188v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection is critical for safe deployment of medical AI systems. Recently, test-time adaptation (TTA) has emerged as a new par

arXiv:2609.07981v1 Announce Type: cross Abstract: Audio provenance attribution - which system produced a synthetic utterance - is reported at near-ceiling accuracy on clean benchmarks, yet audio reach

arXiv:2609.06133v1 Announce Type: new Abstract: Sequential inference on small devices requires a model to retain useful history without repeatedly processing a long input record. A Recurrent Tsetlin M

arXiv:2609.08242v1 Announce Type: new Abstract: Vision-language models (VLMs) demonstrate strong performance across compositional reasoning benchmarks, which require reasoning over semantic perturbati

arXiv:2609.05539v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have recently made strong progress in vision-language reasoning, yet their performance often degrades as genera

arXiv:2609.08705v1 Announce Type: new Abstract: Table Structure Recognition (TSR) aims to extract the bounding boxes of cells and table structure (e.g., HTML) from table images. Although current appro

arXiv:2609.08896v1 Announce Type: cross Abstract: The rapid growth of multimodal data has intensified the need for question answering (QA) systems capable of reasoning across heterogeneous sources suc

arXiv:2609.06046v1 Announce Type: cross Abstract: We present a modular, high-fidelity simulation framework for the development and benchmarking of flight control strategies in fixed-wing aerial robots

On September 9, 2026 Together AI announced that its new **glm‑5.3 flash** model outperforms OpenAI’s **Claude Fable 5.1** on agentic automation tasks. The benchmark also shows glm‑5.3 flash achieves a

GPT‑6 Astra is OpenAI’s latest flagship model, outperforming GPT‑5.6 across virtually all common benchmarks—most noticeably in math, coding, and especially 3‑D rendering/animation tasks. The architect

arXiv:2412.09529v4 Announce Type: replace Abstract: Radiology, with its heterogeneous modalities, anatomies, and evolving protocols, is a natural yet high-stakes testbed for agentic AI. As LLMs grow m

arXiv:2609.06966v1 Announce Type: new Abstract: Model misalignment, prompt injection, or operator misuse could lead AI agents operating frontier-lab accounts to exfiltrate model weights, poison traini

arXiv:2609.07681v1 Announce Type: new Abstract: Associative Recall (AR) is the cognitive ability to learn and retrieve links between items in memory. In NLP, AR is used as a benchmark for evaluating t

arXiv:2507.15824v2 Announce Type: replace Abstract: Text-to-video (T2V) models now produce striking visuals, yet they routinely violate everyday physics; objects float, tools are misused, and causal s

arXiv:2609.06207v1 Announce Type: new Abstract: Image-to-video generators are often credited with absorbing physical dynamics as implicit world models, a claim the community currently checks with plau

arXiv:2609.08038v1 Announce Type: new Abstract: Smart healthcare monitoring systems require precise action recognition to ensure well-being and timely intervention in critical situations such as falls

arXiv:2609.06530v1 Announce Type: new Abstract: Continual learning is a crucial capability for Graphical User Interface (GUI) agents to adapt to evolving applications while retaining knowledge acquire

arXiv:2609.08618v1 Announce Type: new Abstract: Benchmark scores describe what a checkpoint can do now, but they do not determine how it will respond to the next training episode. We measure this miss

arXiv:2609.05596v1 Announce Type: new Abstract: Can interactive vision-and-language agents learn not just what to say but also extbf{extit{when}} to say it? Current language models rarely plan over wh

arXiv:2609.08342v1 Announce Type: new Abstract: World models take multimodal inputs like text, photos, and diagrams to generate dynamic scenes in accordance with the laws of physics, thus opening a co

This is from the https://huggingface.co/IFM/K2-Horizon-0.9B benchmark image Tried searching for it but couldn't find it, I am guessing is https://huggingface.co/openbmb/MiniCPM5-1B but the numbers do

arXiv:2609.06646v1 Announce Type: new Abstract: Annotators routinely disagree on laughter boundaries and subtle chuckles, yet temporal laughter localization typically evaluates against a single refere

Most of the new ~30B (and now ~10B, thankfully for my GPU) models we see score really high on benchmarks, but I feel like they don't push back on dumb ideas enough. I think most people don't being lik

What is Astra actually supposed to be better at, Because if the answer is “everything,” then I genuinely do not understand what kind of testing was done for writing-heavy use cases. My biggest issue i

Hey guys, After my CPU-only to 96GB VRAM test, I tested Qwen3.8-Flash-Next across llama.cpp, SGLang and FreeToken on the same workstation. This time I wanted to see what changes when you keep the hard

arXiv:2609.04409v1 Announce Type: cross Abstract: Multilingual language models often produce inconsistent answers to semantically equivalent questions across languages, motivating methods to improve c

arXiv:2609.04915v1 Announce Type: new Abstract: Many long-horizon LLM deployments face tight prompt budgets: latency, cost, and context limits make full-context prompting impractical as interaction le

arXiv:2609.04357v1 Announce Type: cross Abstract: Purpose: Increased number of chest radiograph (CXR) scans create a triage bottleneck, queueing urgent examinations behind routine ones. Existing AI to

arXiv:2609.05374v1 Announce Type: new Abstract: Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient traject

arXiv:2609.04959v1 Announce Type: new Abstract: Recent calls for harder machine translation benchmarks have not clarified what difficulty should mean. We argue that one meaningful and currently unmeas

arXiv:2609.04280v1 Announce Type: cross Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what they can do.

arXiv:2609.04444v1 Announce Type: new Abstract: Benchmarks for the side effects an agent causes on the way to a goal already exist, but HarvestBench is the first to put a price on avoiding the side ef

arXiv:2603.14503v2 Announce Type: replace Abstract: Galaxy clusters are powerful probes of astrophysics and cosmology through gravitational lensing: the clusters' mass, dominated by 85% dark matter, d

TL;DR My TAK quant of Qwen 3.8 27b scored 82.81% on reasoning, comparted with 77.34% for the byte matched Unsloth UD IQ2_S and 83.59% for BF16. Edit: Some of you have tried coding with this reasoning-

arXiv:2609.05202v1 Announce Type: cross Abstract: Multi-modal learning has demonstrated strong potential in medical applications by integrating heterogeneous data sources such as medical imaging, clin

arXiv:2609.05324v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have shown promising progress in language-conditioned robotic manipulation. However, existing datasets and benchma

TL;DR: switching KV cache to f16 may give a boost in speed if using MTP and ngrams. I have a self-built 'AI mega-cluster' with 2x P40s on a cheap Chinese motherboard and a Xeon CPU (around $1,100 to b

This comparison was requested by a few people, and certainly we were all eager to see the final results. The comparison had taken 11 days and the GPU was crunching numbers for ~167 hours. We've been c

So after all my work, yeah, Raymond did it better, so I ported his work over, extended it turboX, extended it multiple other models (he had only Qwen models), and benchmarked the crap out of it to mak

I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipeline (HVAC industry stuff, lots of long-context retrieval and structured extraction). I have been wa

Note this is separate from privacy - you can make sure your data is not used for training by turning that off in OpenAI and Claude. You can also use an open weights model, but as they continue to impr

arXiv:2412.06494v2 Announce Type: replace Abstract: Background. Federated learning (FL) has gained wide popularity as a collaborative learning paradigm enabling collaborative AI in sensitive healthcar

arXiv:2609.03337v1 Announce Type: new Abstract: Every engineered fill is specified by a maximum dry density and an optimum moisture content. Each determination needs a full Proctor test. Published cor

arXiv:2609.03258v1 Announce Type: new Abstract: Parking spot classification is a fundamental task in intelligent transportation systems, yet most deep learning approaches rely on large amounts of anno

arXiv:2606.09655v2 Announce Type: replace Abstract: Despite remarkable progress in machine translation (MT), non-AI communities have raised growing concerns about MT systems, suggesting a noticeable g