Knowledge catalogue
arXiv:2609.03580v1 Announce Type: new Abstract: The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but u
After Qwen3.8 27B came out, I decided to benchmark the models that could fit in my GPU (RTX 5080) on my actual code (C code), the results were not completely unexpected but some quants were definitely
arXiv:2609.02954v1 Announce Type: new Abstract: Identifying the issues disputed between litigating parties is a crucial component of real-world litigation. However, legal issues remain comparatively u
arXiv:2609.03507v1 Announce Type: cross Abstract: Tracking depression from multi-session counseling dialogues requires estimating both current symptom severity and how it changes across sessions. Yet
arXiv:2609.03690v1 Announce Type: new Abstract: The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advanc
arXiv:2609.04151v1 Announce Type: new Abstract: Generative image models can now produce high-quality images, follow complex instructions, and support precise edits, but they still struggle to preserve
arXiv:2609.04007v1 Announce Type: new Abstract: Despite strong performance on held-out electroencephalography (EEG) data, seizure detectors may fail under real-world acquisition variability, artifacts
arXiv:2609.03804v1 Announce Type: new Abstract: Communities are fundamental spatial units that shape urban form and social life. Whether a residential compound is spatially open or enclosed affects mo
arXiv:2601.19956v2 Announce Type: replace-cross Abstract: As Speech Language Models (SLMs) transition from personal devices to shared, multi-user environments such as smart homes, a new challenge emer
arXiv:2609.02831v1 Announce Type: new Abstract: Modern cameras transform RAW sensor measurements into sRGB images through an image signal processor (ISP). We benchmark two placements for blind restora
arXiv:2603.19539v2 Announce Type: replace-cross Abstract: We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assess
arXiv:2507.21815v2 Announce Type: replace Abstract: Millions of individuals' well-being are challenged by the harms of substance use. Harm reduction as a public health strategy provides non-judgementa
arXiv:2609.01772v1 Announce Type: new Abstract: Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vis
arXiv:2609.01584v1 Announce Type: new Abstract: Vehicle attribute analysis is a key component of Intelligent Transportation Systems (ITS), supporting applications such as vehicle identification, traff
arXiv:2609.01603v1 Announce Type: cross Abstract: Evaluating software engineering agents on realistic benchmarks is costly, since each task may require multi-step code exploration, modification, and t
arXiv:2609.01027v1 Announce Type: new Abstract: Out-of-distribution (OOD) detection predicts whether a test image belongs to none of the predefined classes. To evaluate this task, benchmarks need imag
arXiv:2601.16690v2 Announce Type: replace Abstract: We introduce EMemBench, a programmatic benchmark generator for evaluating long-term episodic memory of agents through interactive games. Rather than
arXiv:2503.04822v2 Announce Type: replace-cross Abstract: Graph neural networks (GNNs) have demonstrated success in modeling relational data primarily under the assumption of homophily. However, many
Kaitchup just posted results of his benchmarks for Qwen3.8 27B for quants from different labs, Q4 to Q1, . All the details are hidden behind the paywall, but high level result is visible and looks lik
MLCommons announced the release of MLPerf Storage v3.0 benchmark results, expanding the suite to cover the full spectrum of AI storage workloads, including an S3 object‑storage access layer in additio
arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However,
arXiv:2603.23938v2 Announce Type: replace Abstract: Most testbeds for omni-modal models assess multimodal understanding via textual outputs, leaving it unclear whether these models can properly speak
arXiv:2608.28663v1 Announce Type: new Abstract: Reliable individual cattle identification supports disease surveillance, vaccination records, breeding management, and livestock insurance. Although the
Qdrant has released **Qdrant‑Fineweb‑10B**, a public benchmark dataset comprising dense and sparse embeddings for 10 billion documents from the FineWeb corpus and exact results for 120,000 queries to
arXiv:2608.29481v1 Announce Type: new Abstract: Pediatric serious illness communication (SIC) is critically important, yet scalable communication training for clinicians remains limited. Compared with
arXiv:2605.05726v2 Announce Type: replace Abstract: As LLM agents are increasingly deployed with large libraries of reusable skills, selecting the right skill for a user request has become a critical
arXiv:2608.28266v1 Announce Type: new Abstract: Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still la
arXiv:2509.26536v3 Announce Type: replace Abstract: We introduce OceanGym, the first comprehensive benchmark for ocean underwater embodied agents, designed to advance AI in one of the most demanding r
I have a 2 DGX Spark setup recently and I have been happily running Deepseek V4 Flash 0731. Since the release of GLM5.3 Flash and Qwen 3.8 Flash Next this week, a lot of folks are still waiting to see
I saw someone here posted about getting a Tenstorrent QuietBox 2, and I wanted to look into it the hardware. It's very difficult to find any benchmarks, but I managed to find some from an employee. Th
arXiv:2608.26713v1 Announce Type: cross Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have extended Image Aesthetic Assessment (IAA) beyond scalar scores toward interpretable c
arXiv:2608.26434v1 Announce Type: new Abstract: Code-switching is pervasive in bilingual African conversation, yet most ASR systems assume monolingual input and are evaluated on curated monolingual be
arXiv:2608.26592v1 Announce Type: new Abstract: Following clinical decision pathways (CDPs) defined by clinical practice guidelines is essential for safe and reliable medical decision-making. However,
arXiv:2608.26433v1 Announce Type: new Abstract: Data-driven prognostics and health management has emerged as a key enabler for Industry 4.0, yet the development of robust remaining useful life (RUL) e
Perplexity AI has released its new language model GLM 5.3 on Perplexity Computer. Designed for long‑context, multimodal agent workloads, GLM 5.3 outperformed the previous GLM 5.2 on the company’s WAND
Using https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 and https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_on_1x_rtx_pro_6000_171_ts_c1_428/ As usual, all the d
arXiv:2608.25934v1 Announce Type: cross Abstract: Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a
arXiv:2608.25005v1 Announce Type: new Abstract: The imperfective paradox provides a useful test of compositional semantic analysis. Recent work constructs an NLI benchmark and reports that models freq
arXiv:2608.17889v1 Announce Type: cross Abstract: Visually rich documents encode relevance through language, layout, structured visual elements, and corpus context, yet retrieval is typically evaluate
Great paper on why agent leaderboard comparisons are hard to trust. It's on the hot topic of how much of an agent benchmark score actually belongs to the harness. The harness is the layer between the
arXiv:2602.06268v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, pr
We built one of the most comprehensive benchmarks for document extraction, and evaluated it across a lot of different systems: ✅ one-shot frontier VLMs ✅ frontier VLMs + coding agent harnesses ✅ one-s
arXiv:2608.21880v1 Announce Type: new Abstract: Bangla large language model (LLM) safety is difficult to evaluate with English-centric or standard-script benchmarks because Bangla users routinely writ
arXiv:2508.18646v3 Announce Type: replace Abstract: Despite their rapid advancement, large language models (LLMs) suffer from a critical disconnect between benchmark scores and real-world utility. Cur
arXiv:2608.23035v1 Announce Type: new Abstract: As on-device LLM agents evolve into personal copilots, the mobile operating system has become a key testbed for this paradigm, making rigorous capabilit
arXiv:2608.22279v1 Announce Type: cross Abstract: Recent vision language models (VLMs) have achieved strong progress in video understanding. However, most existing video QA research and benchmarks sti
arXiv:2608.21249v1 Announce Type: new Abstract: While recent large language models (LLMs) have achieved promising results on individual patent drafting tasks, they fundamentally fail to investigate th
arXiv:2606.18847v2 Announce Type: replace Abstract: To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing lon
WARNING: I think I made a mistake with omlx, and its results below seem to be without any kind of acceleration. I am rerunning them with Lightning MTP and ANE support for prefill. Will update the post
arXiv:2608.20318v1 Announce Type: new Abstract: Recursive self-improvement (RSI) asks whether an AI system can improve the process that produces AI systems, so that the next system inherits the improv
arXiv:2608.20204v1 Announce Type: new Abstract: Legal work, with its heavy reliance on processing large amounts of text, is often considered one of the domains most exposed to the use of LLMs. Contrac
arXiv:2608.19297v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) excel in medical applications, most of them favor static images or short-term signals. In the critical fi
arXiv:2608.20312v1 Announce Type: new Abstract: The capability to perceive and synthesize human-human interactions is fundamental to developing intelligent digital human systems. However, existing dat
arXiv:2608.19901v1 Announce Type: cross Abstract: Agent Skills extend LLM agents with reusable instruction packages that may also include scripts, resources, and service configuration. This creates a
arXiv:2608.19741v1 Announce Type: new Abstract: Recent agent benchmarks increasingly ground evaluation in executable environments, from code repair to web navigation, app APIs, and function calling. Y
arXiv:2511.22842v3 Announce Type: replace-cross Abstract: Causal machine learning (Causal ML) aims to answer 'what if' questions using machine learning algorithms, making it a promising tool for high-
arXiv:2608.18097v1 Announce Type: new Abstract: We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet corp
arXiv:2608.17671v1 Announce Type: cross Abstract: Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools report success rates above 80% on their respective
arXiv:2608.17293v1 Announce Type: cross Abstract: Existing research on irregular time-series forecasting has primarily focused on model design, while evaluation metrics remain insufficiently studied.
arXiv:2607.04784v1 Announce Type: cross Abstract: Defining the reasoning boundaries and ensuring the reliability of Large Reasoning Models (LRMs) remains a critical challenge. Current benchmarks prima