Search: “evaluation”

Adaptive Generation of Bias-Eliciting Questions for LLMs

arXiv:2510.12857v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide. Despite th

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Beyond wheelchairs and blindfolds: Investigating disability stereotypes in T2I models with INCLUDE-BENCH

DGX agent

arXiv:2607.08515v1 Announce Type: new Abstract: Text-to-image (T2I) models have been shown to exhibit social biases. Prior work has mainly focused on gender, skin tone, and cultural representation wit

model-releasesarxiv-cs-cv

10 Jul 2026

CausalDS: Benchmarking Causal Reasoning in Data-Science Agents

arXiv:2607.08093v1 Announce Type: new Abstract: Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant b

Diagnosing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

model-releasesarxiv-cs-lg

arXiv:2606.17093v2 Announce Type: replace Abstract: Learning-based single-shot fringe projection profilometry (FPP) has been studied almost entirely at close range, and the networks used are evaluated

Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?

arXiv:2607.08514v1 Announce Type: new Abstract: Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models

researcharxiv-cs-cv

folks who’ve tried Claude Design — how was the experience? what works well and what could be better?

Claude Design appears to be a design-focused tool or feature within Claude that users can interact with; this post solicits feedback from people who have tried it, asking them to evaluate what aspects

model-releasesthariq--x

Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing

arXiv:2607.07953v1 Announce Type: cross Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at

Persona Cartography: Charting Language Model Personality Traits in Weight Space

arXiv:2607.07916v1 Announce Type: new Abstract: Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decompo

safetyarxiv-cs-ai

Industry

Real-time dental image verification with Amazon SageMaker AI at Henry Schein One

This post describes how Henry Schein One closed that gap by building Image Verify, an AI-powered quality verification system on Amazon SageMaker AI that evaluates dental X-ray quality at the point of

industryaws-ml-blog

Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling

arXiv:2607.08757v1 Announce Type: cross Abstract: Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own tr

researcharxiv-cs-lg

Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning

arXiv:2607.08444v1 Announce Type: cross Abstract: In this paper, we study quantile-based distributional reinforcement learning from the perspective of statistical efficiency. We focus on distributiona

safetyarxiv-cs-lg

Applications

The complexities of patient-centred conversational artificial intelligence

arXiv:2607.08625v1 Announce Type: new Abstract: Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment. However, chatbot development and e

applicationsarxiv-cs-ai

The most important thing about Grok Build and the 4.5 release is that it is genuinely so useful for real-world work

The most important thing about Grok Build and the 4.5 release is that it is genuinely so useful for real-world work Grok 4.5 just topped Perplexity’s WANDR orchestrator evaluation It scored higher tha

agentselon-musk--x

Industry

Try Grok 4.5 in Perplexity

Try Grok 4.5 in Perplexity Grok 4.5 is now available as an orchestrator model in Computer for Consumer Pro and Max subscribers. We evaluated it against five other orchestrator configurations on WANDR.

industryelon-musk--x

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

arXiv:2607.08065v1 Announce Type: new Abstract: LLM-as-judge (Zheng et al., 2023) is increasingly the default for evaluating AI systems in enterprise pipelines, often scaled to ensembles (Verga et al.

BubbleSH: A Dataset of Rising Bubbles with Deformable Interfaces

model-releasesarxiv-cs-lg

arXiv:2607.07275v1 Announce Type: new Abstract: Bubbly flows exhibit complex multiscale dynamics, with deformable bubbles interacting through the surrounding liquid and giving rise to strongly coupled

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

arXiv:2607.07601v1 Announce Type: cross Abstract: Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize co

safetyarxiv-cs-ai

Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study

arXiv:2607.06603v1 Announce Type: cross Abstract: The notion of algorithmic fairness has been actively explored from various aspects of fairness, such as counterfactual fairness (CF) and group fairnes

safetyarxiv-cs-ai

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

arXiv:2607.06641v1 Announce Type: cross Abstract: Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hall

Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

arXiv:2607.07695v1 Announce Type: new Abstract: We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task

Manual, Joystick, or Haptic Control? An In Vitro Comparison of Navigation Strategies for Robotic Interventional Neuroradiology Procedures

arXiv:2607.07253v1 Announce Type: new Abstract: Objective: To evaluate robotic controller interfaces for interventional neuroradiology procedures in-vitro incorporating a force-sensing platform to ass

safetyarxiv-cs-ro

ppl often ask what i'm looking at for investing. here's what's showing up in my pipeline today

Yohei Nakajima shares his current investment pipeline and opportunities he's evaluating, providing insight into the types of ventures and sectors capturing his attention as an investor. The post likel

agentsyohei-nakajima--x

Practicing with Language Models Cultivates Human Empathic Communication

arXiv:2603.15245v2 Announce Type: replace Abstract: Empathy is central to human connection, yet people often struggle to express it effectively. In blinded evaluations, large language models (LLMs) ge

researcharxiv-cs-cl

Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees

arXiv:2405.16668v2 Announce Type: replace Abstract: Adversarial Imitation Learning (AIL) faces challenges with sample inefficiency because of its reliance on sufficient on-policy data to evaluate the

safetyarxiv-cs-lg

Tools

See how every model compares: http://cursor.com/evals

Cursor shared a link to their evals page that provides a comprehensive comparison of different AI models' performance and capabilities. The page likely displays benchmarks, metrics, or evaluation resu

toolscursor--x

// The Harness Effect // (bookmark it) Now more that ever pay very close attention to the orchestration harness and its effect on costs and …

// The Harness Effect // (bookmark it) Now more that ever pay very close attention to the orchestration harness and its effect on costs and performance. This study ran 22 evaluation tasks on six found

model-releasesdair-ai--x

Trexplorer Super: Topologically Correct Centerline Tree Tracking of Tubular Objects in CT Volumes

arXiv:2507.10881v2 Announce Type: replace Abstract: Tubular tree structures, such as blood vessels and airways, are essential in human anatomy and accurately tracking them while preserving their topol

researcharxiv-cs-cv

What Predicts Correctness in Text-to-SQL? A Selective-Prediction Study

arXiv:2607.06799v1 Announce Type: cross Abstract: Evaluating uncertainty in AI-generated SQL queries requires estimating whether a query is correct, where correct means it executes to the same result

BabyVision: Visual Reasoning Beyond Language

model-releasesarxiv-cs-cl

arXiv:2601.06521v2 Announce Type: replace-cross Abstract: While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic

Industry

Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase

This Databricks blog post evaluates the performance and capabilities of coding agents when applied to real-world scenarios involving their own multi-million line codebase, likely assessing metrics suc

industrydatabricks

Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis

arXiv:2607.05842v1 Announce Type: cross Abstract: Large language model (LLM)-assisted software security operates at a difficult boundary: the vulnerability-analysis terminology needed for legitimate c

Hardware

BitLogic: Training Framework for Gradient-Based FPGA-Native Neural Networks

arXiv:2602.07400v2 Announce Type: replace Abstract: Gradient-based LUT- and logic-gate-based neural networks (LUTNet, LogicNets, DiffLogic, PolyLUT, NeuraLUT, WARP-LUT, DWN, LILogicNet, LightLUT) repl

hardwarearxiv-cs-lg

Bridging Diffusion Pruning and Step Distillation with Teacher-Aligned Repair

arXiv:2607.06335v1 Announce Type: new Abstract: Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. E

safetyarxiv-cs-cv

Finding H. pylori in the Fine Print: Evidence-Linked Multi-Agent Case Finding from Gastric Biopsy Reports

arXiv:2607.06435v1 Announce Type: new Abstract: Data from Singapore indicated that about 31% of the population had evidence of Helicobacter pylori infection. Persistent H. pylori infection is associat

agentsarxiv-cs-ai

Foundation Models for Automatic CAD Generation

arXiv:2607.05573v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) enable the automatic generation of parametric 3D designs from natural-

From Global to Granular: Revealing IQA Model Performance via Correlation Surface

arXiv:2601.21738v2 Announce Type: replace-cross Abstract: Evaluation of Image Quality Assessment (IQA) models has long been dominated by global correlation metrics, such as Pearson Linear Correlation

researcharxiv-cs-ai

From Voting to Agent Collaboration: Answer-Type-Aware LLM Pipelines for BioASQ 14b

arXiv:2607.06452v1 Announce Type: cross Abstract: Biomedical question answering requires not only accurate extraction of information from scientific literature but also reliable integration of evidenc

agentsarxiv-cs-ai

Grok 4.5 true usefulness is excellent

model-releaseselon-musk--x

Grok 4.5 true usefulness is excellent We obtained early access to evaluate @SpaceXAI's newest Grok 4.5 model on GDPval+: real professional deliverables (from a variety of economic sectors and occupati

Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context

arXiv:2607.05880v1 Announce Type: cross Abstract: Imaging demand is growing faster than the radiology workforce can expand, and reporting backlogs cannot be resolved through training and recruitment a

researcharxiv-cs-ai

Leveraging Neural Graph Compilers in Machine Learning Research for Edge-Cloud Systems

arXiv:2504.20198v2 Announce Type: replace-cross Abstract: This work presents a comprehensive evaluation of neural network graph compilers across heterogeneous hardware platforms, addressing the critic

researcharxiv-cs-lg

LingDT-VL-OCR: Structure-Aware Document-Level Parsing with Fine-Grained Visual Reference

model-releasesarxiv-cs-cv

arXiv:2603.11044v2 Announce Type: replace Abstract: In this paper, we propose LingDT-VL-OCR, a document parsing system tailored to financial-domain documents, transforming ultra-long financial PDFs in

LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability

arXiv:2607.06157v1 Announce Type: cross Abstract: Deliberation plays a crucial role in collaboration; when humans work together, they naturally engage in communication to align information and reach a

No-Regret Gaussian Process Optimization of Time-Varying Functions

arXiv:2512.00517v3 Announce Type: replace-cross Abstract: Sequential optimization of black-box functions from noisy evaluations has been widely studied, with Gaussian Process bandit algorithms such as

researcharxiv-cs-lg

PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails

arXiv:2607.05910v1 Announce Type: cross Abstract: Image guardrails are typically trained and evaluated under a fixed safety policy, implicitly treating safety as an intrinsic property of an image. Rea

PolyWorkBench: Benchmarking Multilingual Long-Horizon LLM Agents

arXiv:2607.06008v1 Announce Type: new Abstract: Large language model (LLM) agents have shown strong performance in long-horizon tasks that require planning, tool use, and interaction with external env

Quantifying Frontier LLM Capabilities for Container Sandbox Escape

arXiv:2603.02277v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, cr

Reward-Density Heuristic for Dynamic Multi-Vehicle Routing: Performance and Computational Efficiency

arXiv:2607.06066v1 Announce Type: new Abstract: The Vehicle Routing Problem (VRP) and its variants represent some of the most practically consequential optimization challenges in modern logistics and

agentsarxiv-cs-ai

Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations

arXiv:2504.05294v3 Announce Type: replace Abstract: Chain-of-thought explanations are widely used to inspect the decision process of large language models (LLMs) and to evaluate the trustworthiness of

safetyarxiv-cs-cl