Search: “benchmark”

V4-Flash-0731 - vibes after first weekend of use

Spent way too much time with V4-Flash-0731 this weekend and wanted to share my vibes as briefly as possible. I sent it through a bit of real-work and some of my personal benchmarks. My quick thoughts

3 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DeepSeek-V4-Flash-0731: When Low is higher than High

DGX agent

I decided to test a few questions against DeepSeek-V4-Flash-0731. Locally, I was running Unsloth's UD-Q2_K_XL quant. After I saw the surprising shape of the results, I tested against DeepSeek's offici

model-releasesr-localllama

2 Aug 2026

Encrypted Clouds?

I love the progress happening on open models but I feel like it is kind of getting clear that hardware to run good sized models is completely unaffordable for me right now. I know that you all love Qw

2 Aug 2026

Agents

How do you test your setup?

We all have been there, tinkering around with models is fun but we rarely do it with research precision and issues are often subtle and hard to reproduce. There are a lot of benchmarks but running the

agentsr-localllama

2 Aug 2026

Parlor v2: best-effort fully local GPT-Live clone on an M3 Pro

GPT-Live is so good that I use it almost every day. I've been wanting to replicate it since it was released. My first attempt was to fine-tune Gemma 4 12B to behave like a full-duplex model. Something

2 Aug 2026

DeepSeek-V4-Flash-0731: Models you can run locally now have the intelligence score of the top frontier model from March 2026

March 6th, 2026 the highest intelligence index score was 51 for frontier models. deepseek-ai/DeepSeek-V4-Flash-0731 that has an intelligence score of 50. If these benchmarks are accurate, models avail

1 Aug 2026

A novel k-means clustering approach using two distance measures for Gaussian data

arXiv:2511.17823v2 Announce Type: replace Abstract: Clustering algorithms have long been the topic of research, representing the more popular side of unsupervised learning. Since clustering analysis i

Beyond Sentiment: Structured Information Extraction from Financial News

arXiv:2607.28496v1 Announce Type: new Abstract: Financial sentiment analysis has become a standard component in news-driven stock prediction, yet it reduces rich, multi-dimensional news articles to a

Can LVLMs Uncover the Truth Behind Visual Illusions? An Analysis of Perceptual and Reasoning Capabilities

arXiv:2607.27747v1 Announce Type: new Abstract: Large Vision Language Models have integrated reasoning capabilities, elevating cognitive performance to new levels. However, existing evaluations either

Agents

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

arXiv:2607.28591v1 Announce Type: cross Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a r

agentsarxiv-cs-cl

Dimensionality and Measurement Precision in HLE's Multiple-Choice Subset

arXiv:2607.27420v1 Announce Type: cross Abstract: Humanity's Last Exam (HLE) is widely used to evaluate frontier language models. HLE organizes its questions into eight subject-domain categories, whos

FinanceHarness: Autonomous Financial Deep Research Framework

arXiv:2607.27853v1 Announce Type: new Abstract: Powered by advances in LLMs and autonomous agents, deep research has become one of the most widely adopted agentic products. However, most deep research

IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurations

arXiv:2607.26075v1 Announce Type: cross Abstract: We present IDP AutoOpt, an autonomous LLM agent that discovers high-performing configurations for intelligent document processing (IDP) pipelines. Tun

IFCMemoryBench: Evaluating Long-Term Memory of LLM-Based Agents in BIM Information Retrieval

arXiv:2607.26072v1 Announce Type: cross Abstract: Long-term memory is becoming a core capability of LLM-based agents, but existing evaluations largely test conversational recall in open-domain or pers

Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory

arXiv:2607.27919v1 Announce Type: new Abstract: Decoder-only language models entangle long-term memory and reasoning in a single parameter set, making it difficult to scale memory capacity independent

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

arXiv:2511.12449v3 Announce Type: replace Abstract: Recent Multimodal Large Language Models (MLLMs) have significantly advanced e-commerce product understanding. However, they still face three challen

MRD: Using Physically Based Differentiable Rendering to Probe Vision Models for 3D Scene Understanding

arXiv:2512.12307v5 Announce Type: replace Abstract: While deep learning methods have achieved impressive success in many vision benchmarks, it remains difficult to understand and explain the represent

researcharxiv-cs-cv

ORCA-bench: How Ready Are Language Model Agents for Oncall?

arXiv:2607.28545v1 Announce Type: new Abstract: Large language models can write, patch, and search code, but oncall root cause analysis (RCA) demands something different: reasoning over noisy metrics,

Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents

arXiv:2607.28227v1 Announce Type: cross Abstract: GUI agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision a

SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation

arXiv:2607.26313v1 Announce Type: cross Abstract: Agentic systems act, so a defect in the evidence they retrieve becomes a wrong action with a currency cost. The most dangerous enterprise defects are

Selecting Open-Weight Language Models for Zero-Shot Intent Classification: A Systematic Evaluation of 41 Models

arXiv:2607.27421v1 Announce Type: new Abstract: Intent classification is a core component of task-oriented dialogue systems, yet practitioners have limited systematic guidance for selecting deployable

SenseNova U1.5 Lite preview just dropped

SenseNova released U1.5-Lite-Preview Benchmarks: Qwen-Image-Bench from 47.14 to 55.20. ImgEdit-Bench from 3.90 to 4.37. GEdit-Bench-en from 7.47 to 8.17. Key updates: 4K native generation with better

Space2Ground 2.0: A Multi-Source Dataset and Framework for Agricultural Monitoring through Fusion of Street-Level and Satellite Imagery

arXiv:2607.28247v1 Announce Type: new Abstract: Accurate and scalable parcel-level agricultural monitoring remains challenging because satellite Earth Observation alone provides only an overhead persp

StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agents

arXiv:2607.26314v1 Announce Type: cross Abstract: Stealth, the discipline of achieving an objective without revealing your presence, capabilities, or collected intelligence, is what separates sophisti

THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model

arXiv:2607.27303v1 Announce Type: cross Abstract: Temporal heterogeneous graphs offer a natural abstraction for dynamic relational systems in which diverse node and relation types co-exist and evolve

TraceCoder: Explainable and Auditable Code Generation with Position-Key Snippet Versioning

arXiv:2607.26307v1 Announce Type: new Abstract: Contemporary LLM-based coding agents produce code as black-box outputs: the rationale behind each line is hidden, the evolution of the code through benc

Try Grok 4.5 http://X.ai/cli

Try Grok 4.5 http://X.ai/cli BREAKING: Grok 4.5 outperforms GPT-5.6 Terra across almost all shared benchmarks on AskClash, leading in ACB, GPQA, SWE-P, and Atlas while also achieving a higher overall

4090 + 5060 Ti + 64GB RAM: 206 t/s on a 35B-A3B, and a 122B at 37 t/s

I've been benchmarking a two-card box for a few weeks and I still can't quite get over some of these numbers, so I'm dumping them here. Box: RTX 4090 (24GB) + RTX 5060 Ti (16GB), i9-13900K, 64GB DDR5.

And Grok 4.6 comes out in a week

And Grok 4.6 comes out in a week BREAKING: Grok 4.5 ranked #1 on LaurenBench with a score of 56.9%, ahead of Claude Sonnet 5, GLM 5.2, Claude Opus 5, Kimi K3 and GPT-5.6. The benchmark tests real-worl

Entity Resolution in Practice: Lessons from a Self-Serve Pipeline

arXiv:2607.26298v1 Announce Type: new Abstract: We built and evaluated a self-serve entity resolution (ER) system on six benchmarks spanning 864 to 5M records, and three lessons emerged that are absen

researcharxiv-cs-lg

Gemini Robotics ER 2: powering robotics with video understanding, task orchestration, and multi-robot collaboration

model-releasesgoogle-deepmind

Gemini Robotics ER 2 helps robots reason, collaborate, and solve real-world tasks. It represents a step change in video understanding, tool orchestration, and multi-robot collaboration for robotic app

Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer

arXiv:2607.26582v1 Announce Type: new Abstract: Selecting a zero-shot out-of-distribution (OOD) detector for a new deployment is typically based on benchmark rankings, implicitly assuming that the hig

Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback

arXiv:2607.26094v1 Announce Type: cross Abstract: Reinforcement Learning from Human Feedback (RLHF) is the standard approach for aligning large language models with human preferences, but its quality

Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Text

arXiv:2607.26751v1 Announce Type: new Abstract: State-of-the-art intracortical brain-to-text systems pair a neural-sequence phone decoder with an external language model. Two design axes remain undere

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

arXiv:2607.26326v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models. How

TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees

arXiv:2607.27027v1 Announce Type: new Abstract: Gradient-boosted trees dominate tabular machine learning, yet canonical correlation analysis has always relied on linear or neural encoders. We propose

Two Calls Beat Five Agents: Evaluating Multi-Agent Pipelines Against Self-Refinement for Local Language Models

arXiv:2607.26922v1 Announce Type: new Abstract: Multi-agent LLM pipeline systems break down the task among multiple roles for better reasoning, but are benchmarked mainly with large-scale commercial m

BREAKING: Grok 4.5 ranked #1 on LaurenBench with a score of 56.9%, ahead of Claude Sonnet 5, GLM 5.2, Claude Opus 5, Kimi K3 and GPT-5.6. Th…

BREAKING: Grok 4.5 ranked #1 on LaurenBench with a score of 56.9%, ahead of Claude Sonnet 5, GLM 5.2, Claude Opus 5, Kimi K3 and GPT-5.6. The benchmark tests real-world AI agents across conversation,

BREAKING: SpaceXAI's newly released Grok Voice Think Fast 2.0 beats voice models from OpenAI, Google, Alibaba, and DeepSlate in the Artifici…

SpaceXAI has released its new Grok Voice Think Fast 2.0, which on the Artificial Analysis Speech‑to‑Speech benchmark outperformed leading models from OpenAI, Google, Alibaba and DeepSlate. The claim w

DecoEvo: Score-Decoupled Co-Evolution of Solver and Rubric-Generator Skills in Text Space

arXiv:2607.25675v1 Announce Type: new Abstract: Text-space optimization adapts large language models (LLMs) by editing external natural-language artifacts rather than model weights, so the optimized a

GAUGE: Grading Agent-Built Financial Models Without a Golden Answer

arXiv:2607.24889v1 Announce Type: cross Abstract: Financial models combine public disclosures with analyst assumptions to produce forecasts and valuations. While some components can be checked mechani

LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation

arXiv:2607.24780v1 Announce Type: new Abstract: Evaluating frontier LLMs is challenging: static benchmarks suffer from contamination and saturation -- leaving users unable to distinguish top models an

researcharxiv-cs-ai

ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization

arXiv:2607.25524v1 Announce Type: cross Abstract: Unmanned aerial vehicle (UAV)-satellite cross-view geo-localization matches UAV images against satellite imagery and has achieved impressive accuracy

Towards Real-World RUL Prediction for Aircraft Compressors Under Variable Conditions with Physics-Guided Domain Adaptation

arXiv:2607.24809v1 Announce Type: cross Abstract: Remaining useful life prediction for aircraft centrifugal air compressors in real commercial operations poses challenges that controlled benchmark dat

Bayesian Complete-Pooling in Cross-Subject Classification for Motor Imagery Electroencephalogram

arXiv:2607.22980v1 Announce Type: new Abstract: Brain-computer interfaces (BCIs) have long sought calibration-free operation, but classifiers are typically benchmarked by discrimination alone, blind t

researcharxiv-cs-lg

Characterizing Arbitrary Lindbladian Dynamics with a Few Pauli Measurements

arXiv:2607.23044v1 Announce Type: cross Abstract: Quantum devices are open systems whose dynamics interleave coherent evolution with dissipation, and benchmarking, error mitigation, and error correcti

researcharxiv-cs-lg

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

arXiv:2607.24743v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundam

ConsistencyGate: Preventing Memory Contamination in LLM Agents via Self-Consistency Admission Control

arXiv:2607.22962v1 Announce Type: new Abstract: LLM agents that operate over many turns accumulate facts in an external memory store and reuse them as premises for downstream reasoning. A hallucinated