Knowledge catalogue
arXiv:2605.18498v1 Announce Type: cross Abstract: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balanci
arXiv:2502.04055v2 Announce Type: replace Abstract: Current evaluations of synthetic tabular data mainly focus on how well joint distributions are modeled, often overlooking the assessment of their ef
arXiv:2605.18663v1 Announce Type: new Abstract: As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficulty: escalating knowledge demands (GPQA, HLE) or remo
arXiv:2605.17932v1 Announce Type: cross Abstract: Prompt compression reduces inference cost and context length in large language models, but prior evaluations focus primarily on autoregressive archite
arXiv:2605.14488v1 Announce Type: new Abstract: Large Language Models (LLMs) augmented with Retrieval-Augmented Generation (RAG) techniques are revolutionizing applications across multiple domains, su
arXiv:2605.10186v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into legal drafting and research workflows, where incorrect citations or fabricated precedent
arXiv:2605.10820v1 Announce Type: new Abstract: Scientific discovery is fundamentally a resource-constrained process that requires navigating complex trade-offs between the quality and quantity of mea
arXiv:2605.08437v1 Announce Type: cross Abstract: Existing benchmarks for legal AI focus primarily on tasks where LLMs must produce legal arguments or documents, yet the capacity to judge such argumen
arXiv:2605.07926v1 Announce Type: new Abstract: As LLM-based agents increasingly rely on external tools, it is important to evaluate their ability to sustain tool-grounded reasoning beyond familiar wo
arXiv:2605.07046v1 Announce Type: cross Abstract: Evaluation of large language models (LLMs) is increasingly critical, yet standard benchmarking methods rely on average accuracy, overlooking both the
arXiv:2605.07251v1 Announce Type: new Abstract: Large Language Models (LLMs) have become increasingly capable as tool-using agents, with benchmarks spanning diverse general agentic tasks. Yet rigorous
arXiv:2512.15567v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge and
arXiv:2605.07675v1 Announce Type: new Abstract: We introduce FactoryBench, a benchmark for evaluating time-series models and LLMs on machine understanding over industrial robotic telemetry. Q&A pairs
arXiv:2605.04018v1 Announce Type: new Abstract: Reasoning-intensive retrieval aims to surface evidence that supports downstream reasoning rather than merely matching topical similarity. This capabilit
arXiv:2409.14500v5 Announce Type: replace Abstract: Although data that can be naturally represented as graphs is widespread in real-world applications across diverse industries, popular graph ML bench
arXiv:2604.27470v1 Announce Type: new Abstract: Millions of clinicians use ChatGPT to support clinical care, but evaluations of the most common use cases in model-clinician conversations are limited.
arXiv:2604.25359v1 Announce Type: new Abstract: Large Language Models are increasingly being deployed to extract structured data from unstructured and semi-structured sources: parsing invoices, medica
arXiv:2509.20374v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated strong performance across general NLP tasks, but their utility in automating numerical experime
arXiv:2406.10185v2 Announce Type: replace Abstract: Large Vision Language Models (LVLMs) are increasingly integral to healthcare applications, including medical visual question answering and imaging r
arXiv:2604.23347v1 Announce Type: new Abstract: We present a comprehensive evaluation of Large Language Models (LLMs) on Computer Science (CS) Data Structure examination questions. Our work introduces
arXiv:2604.23813v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabili
Great paper on improving proactive agents. (bookmark it) Proactive agents act before you do. But how do you evaluate something that's supposed to anticipate needs you haven't expressed? This work intr
arXiv:2604.21006v1 Announce Type: new Abstract: We introduce Deep FinResearch Bench, a practical and comprehensive evaluation framework for deep research (DR) agents in financial investment research.
arXiv:2604.21916v1 Announce Type: new Abstract: As frontier language models attain near-ceiling performance on static mathematical benchmarks, existing evaluations are increasingly unable to different
arXiv:2508.17761v3 Announce Type: replace Abstract: In safety-critical applications data-driven models must not only be accurate but also provide reliable uncertainty estimates. This property, commonl
arXiv:2604.18660v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in education, yet their default helpfulness often conflicts with pedagogical principles. Prior work
arXiv:2604.19510v1 Announce Type: new Abstract: Variability in illumination is a primary factor limiting deep learning robustness for field-based plant disease detection. This study evaluates Histogra
arXiv:2604.19526v1 Announce Type: cross Abstract: Cross-site scripting (XSS) remains a persistent web security vulnerability, especially because obfuscation can change the surface form of a malicious
arXiv:2604.19092v1 Announce Type: cross Abstract: Recent advances in large-scale video world models have enabled increasingly realistic future prediction, raising the prospect of leveraging imagined v
arXiv:2604.19405v1 Announce Type: new Abstract: Automatic evaluators such as reward models play a central role in the alignment and evaluation of large vision-language models (LVLMs). Despite their gr
arXiv:2510.19028v3 Announce Type: replace Abstract: As LLMs are increasingly deployed in real-world interactions, their social reasoning in interpersonal communication becomes critical. To explore the
arXiv:2508.17458v2 Announce Type: replace Abstract: Verbal multiword expressions (VMWEs) remain difficult for machine translation because their meanings are often not recoverable from their component
arXiv:2602.06663v2 Announce Type: replace Abstract: Unified multimodal models (UMMs) have shown impressive capabilities in generating natural images and supporting multimodal reasoning. However, their
arXiv:2510.09351v2 Announce Type: replace Abstract: While Small Language Models (SLMs) have demonstrated promising performance on an increasingly wide array of commonsense reasoning benchmarks, curren
EDINET-Bench is a benchmark dataset that evaluates large language models on complex financial analysis tasks using Japanese financial statements from the EDINET database. The benchmark tests LLM capab
arXiv:2604.16279v1 Announce Type: new Abstract: Large Language Models (LLMs) have the potential to accelerate small molecule drug design due to their ability to reason about information from diverse s
arXiv:2604.06484v2 Announce Type: replace Abstract: Cultural values are expressed not only through language but also through visual scenes and everyday social practices. Yet existing evaluations of cu
arXiv:2510.09536v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in multilingual, real-world applications with user inputs -- naturally introducing typographi
arXiv:2604.12447v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models inherit rich world knowledge from vision-language backbones and acquire executable skills via action demonstrations.
arXiv:2604.13035v1 Announce Type: cross Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) increasingly generate indoor scenes through intermediate structures such as layouts and
arXiv:2604.11304v1 Announce Type: new Abstract: Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-
arXiv:2604.10825v1 Announce Type: new Abstract: We introduce CheeseBench, a benchmark that evaluates large language models (LLMs) on nine classical behavioral neuroscience paradigms (Morris water maze
arXiv:2604.11201v1 Announce Type: cross Abstract: LLM agents now perform strongly in software engineering, deep research, GUI automation, and various other applications, while recent agent scaffolds a
arXiv:2604.09698v1 Announce Type: cross Abstract: The growing ubiquity of Extended Reality (XR) is driving Conversational Recommendation Systems (CRS) toward visually immersive experiences. We formali
arXiv:2509.04334v4 Announce Type: replace Abstract: Geographic reasoning is a fundamental cognitive capability that requires models to infer plausible locations by synthesizing visual evidence with sp
arXiv:2604.09937v1 Announce Type: new Abstract: Healthcare administration accounts for over $1 trillion in annual spending, making it a promising target for LLM-based computer-use agents (CUAs). While
arXiv:2604.11655v1 Announce Type: cross Abstract: The rapid adoption of Large Language Models (LLMs) in interactive systems has enabled the creation of dynamic, open-ended Role-Playing Agents (RPAs).
arXiv:2609.33180v1 Announce Type: cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies on finite
Automated lint: 14 errors, 54840 warnings, 3 info
Automated lint: 12 errors, 54839 warnings, 3 info
arXiv:2503.05061v3 Announce Type: replace Abstract: Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expands, particularly in high-stakes domains such as bus
The intersection of medicine and AI has led to remarkable innovations. However, developers now face the thorny challenge of building robust medical AI tools that have been tested and evaluated on dive
arXiv:2509.22768v3 Announce Type: replace Abstract: We introduce ML2B, the first benchmark for evaluating cross-lingual task comprehension in end-to-end ML pipeline generation by large language models
arXiv:2607.12252v1 Announce Type: new Abstract: Deep research agents are increasingly used to produce long-form financial reports, yet large-scale evaluation remains bottlenecked by the need for human
arXiv:2602.13832v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than
arXiv:2610.21088v1 Announce Type: new Abstract: Test-time search can improve agent performance by exploring alternative actions and using a critic to select promising branches. We study this approach
arXiv:2610.21229v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs
Mistral AI introduced a tool aimed at high‑energy physics that can reconstruct particle collisions, identify particles, flag unexpected signals, and evaluate future detector designs. The announcement
arXiv:2610.20360v1 Announce Type: new Abstract: Gossip, defined as evaluative talk about absent third parties, is widely seen as a uniquely human social practice. This study examines whether comparabl
arXiv:2610.19373v1 Announce Type: new Abstract: Does a language model answer better because its compressed memory contains useful document information? Replacing memory vectors with zeros can lower a