Search: “evaluation”

arXiv:2604.13035v1 Announce Type: cross Abstract: Large Language Models (LLMs) and Vision-Language Models (VLMs) increasingly generate indoor scenes through intermediate structures such as layouts and

arXiv:2604.11304v1 Announce Type: new Abstract: Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-

arXiv:2604.10825v1 Announce Type: new Abstract: We introduce CheeseBench, a benchmark that evaluates large language models (LLMs) on nine classical behavioral neuroscience paradigms (Morris water maze

arXiv:2604.11201v1 Announce Type: cross Abstract: LLM agents now perform strongly in software engineering, deep research, GUI automation, and various other applications, while recent agent scaffolds a

arXiv:2604.09698v1 Announce Type: cross Abstract: The growing ubiquity of Extended Reality (XR) is driving Conversational Recommendation Systems (CRS) toward visually immersive experiences. We formali

arXiv:2509.04334v4 Announce Type: replace Abstract: Geographic reasoning is a fundamental cognitive capability that requires models to infer plausible locations by synthesizing visual evidence with sp

arXiv:2604.09937v1 Announce Type: new Abstract: Healthcare administration accounts for over $1 trillion in annual spending, making it a promising target for LLM-based computer-use agents (CUAs). While

arXiv:2604.11655v1 Announce Type: cross Abstract: The rapid adoption of Large Language Models (LLMs) in interactive systems has enabled the creation of dynamic, open-ended Role-Playing Agents (RPAs).

arXiv:2609.33180v1 Announce Type: cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies on finite

Automated lint: 14 errors, 54840 warnings, 3 info

Automated lint: 12 errors, 54839 warnings, 3 info

arXiv:2503.05061v3 Announce Type: replace Abstract: Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expands, particularly in high-stakes domains such as bus

The intersection of medicine and AI has led to remarkable innovations. However, developers now face the thorny challenge of building robust medical AI tools that have been tested and evaluated on dive

arXiv:2509.22768v3 Announce Type: replace Abstract: We introduce ML2B, the first benchmark for evaluating cross-lingual task comprehension in end-to-end ML pipeline generation by large language models

arXiv:2607.12252v1 Announce Type: new Abstract: Deep research agents are increasingly used to produce long-form financial reports, yet large-scale evaluation remains bottlenecked by the need for human

arXiv:2604.19405v1 Announce Type: new Abstract: Automatic evaluators such as reward models play a central role in the alignment and evaluation of large vision-language models (LVLMs). Despite their gr

arXiv:2602.13832v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than

arXiv:2610.21088v1 Announce Type: new Abstract: Test-time search can improve agent performance by exploring alternative actions and using a critic to select promising branches. We study this approach

arXiv:2610.21229v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs

Mistral AI introduced a tool aimed at high‑energy physics that can reconstruct particle collisions, identify particles, flag unexpected signals, and evaluate future detector designs. The announcement

arXiv:2610.20360v1 Announce Type: new Abstract: Gossip, defined as evaluative talk about absent third parties, is widely seen as a uniquely human social practice. This study examines whether comparabl

arXiv:2610.19373v1 Announce Type: new Abstract: Does a language model answer better because its compressed memory contains useful document information? Replacing memory vectors with zeros can lower a

arXiv:2610.19240v1 Announce Type: new Abstract: Financial disclosures express the underlying relationships of double-entry accounting through unstructured text. While financial LLM benchmarks predomin

arXiv:2610.19165v1 Announce Type: cross Abstract: Many real-world questions demand not a single fact but an organized collection of them, yet existing factual knowledge benchmarks almost exclusively t

arXiv:2505.13546v2 Announce Type: replace-cross Abstract: Automatic prompt generation plays a crucial role in enabling general-purpose multi-agent systems to perform diverse tasks autonomously. Existi

arXiv:2610.17665v2 Announce Type: new Abstract: Can vision-language models turn 2D observations into 3D understanding that supports precise action? This requires inferring object geometry, anticipatin

arXiv:2610.17326v1 Announce Type: new Abstract: Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of

arXiv:2610.17898v1 Announce Type: new Abstract: Sparse autoencoders decompose language model representations into interpretable features, but feature interpretation in this literature remains predomin

arXiv:2610.18098v1 Announce Type: new Abstract: Deleting singular components offers a precise way to remove a capability from a language model, yet the first-order spectral saliency used to rank compo

arXiv:2610.14397v1 Announce Type: new Abstract: Augmenting worker capabilities in the physical world through guidance has been a decades-long goal for AI agents. One key supporting capability is to un

arXiv:2610.17531v1 Announce Type: new Abstract: While it is known that quantization of large language models (LLMs) reduces memory usage via lower-bit weights, studies quantifying the resulting impact

🔎 Announced at Interrupt: AI Search in LangSmith You used to need an evaluator or read hours of conversations to find traces that answer questions such as: 💬 Does the user seemed frustrated?' 💬 Where

New paper from Microsoft and colleagues on generating enterprise data for agents. If you train or evaluate agents on long-running business work, it's worth your time. (bookmark it) MiniCorp is an offi

arXiv:2601.18901v2 Announce Type: replace Abstract: Knowledge probing quantifies how much relational knowledge a language model (LM) has acquired during pre-training. Existing knowledge probes evaluat

arXiv:2610.11050v1 Announce Type: new Abstract: Computer-use agents are capable of completing complex tasks, increasing the use of automatic judges to determine success, either for training or for eva

arXiv:2610.10946v1 Announce Type: new Abstract: Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire

arXiv:2610.11069v1 Announce Type: new Abstract: Large language model (LLM) assistants are being deployed to clinicians across health systems, and judgments about their readiness rest largely on benchm

arXiv:2610.10942v1 Announce Type: new Abstract: Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic ben

arXiv:2610.09272v1 Announce Type: new Abstract: Attention routing requires a signal that predicts the value of attention on the current prefix. We evaluate whether hidden-state extrapolation error, cu

arXiv:2510.13276v2 Announce Type: replace Abstract: The rapid advancement of long-context vision language models (LCVLMs) has led to a significant expansion of their context windows. However, an exten

arXiv:2610.02549v1 Announce Type: new Abstract: Time and event expression extraction are fundamental temporal reasoning tasks, but the problem remains difficult due to annotation ambiguity, domain sen

arXiv:2505.21870v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) generally enhances large language models' (LLMs) ability to solve knowledge-intensive tasks. But RAG coul

AI researchers Zainhas and Parthsareen from Ollama delivered a breakout presentation on October 2, 2026, focusing on evaluating open-source models under real‑world production traffic rather than just

arXiv:2610.00649v1 Announce Type: cross Abstract: Synthetic speech detection is critical for audio security, but performance can degrade when labeled data are scarce and evaluation conditions differ f

arXiv:2510.23191v2 Announce Type: replace Abstract: Predictive benchmarking, evaluating machine learning models based on predictive performance and competitive ranking, is central to machine learning

arXiv:2609.38260v1 Announce Type: cross Abstract: Values such as honesty, autonomy, and confidentiality are often regarded as general principles underpinning AI alignment. However, what it means to ac