Search: “evaluation”

BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning

arXiv:2607.29302v1 Announce Type: cross Abstract: Reliable robot learning requires a world simulator that can predict action consequences before execution on physical hardware, including risky and fai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation

DGX agent

arXiv:2604.05150v2 Announce Type: replace-cross Abstract: We study compiled AI, a paradigm in which large language models generate executable code artifacts during a compilation phase, after which wor

safetyarxiv-cs-ai

3 Aug 2026

Tutorials

Contrastive Learning for Image Complexity Representation

arXiv:2408.03230v2 Announce Type: replace Abstract: Quantifying and evaluating image complexity can be instrumental in enhancing the performance of various computer vision tasks. Supervised learning c

tutorialsarxiv-cs-cv

EarlyDx: An Admission-Anchored Benchmark for Open-Ended Generation of Evidence-Supported ED-Encounter Diagnoses

arXiv:2607.28788v1 Announce Type: new Abstract: Clinical diagnosis at hospital admission must be made rapidly from limited, incomplete evidence. Existing diagnosis-prediction benchmarks are poorly sui

Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaboration

arXiv:2607.29087v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in enterprise settings, yet individual models remain bounded by model-specific capability limitat

Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design

arXiv:2607.28894v1 Announce Type: new Abstract: Computational cognitive modeling seeks to infer latent cognitive mechanisms underlying observed behavior. Bayesian inverse planning provides a principle

Improving scDiffusion with Sparsity-Biased Classifier-Free Guidance

arXiv:2607.29043v1 Announce Type: cross Abstract: Single-cell RNA sequencing (scRNA-seq) has become an essential tool in modern cellular biology, and generating accurate synthetic scRNA-seq data is be

researcharxiv-cs-ai

Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination

arXiv:2607.28947v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to solve complex problems by searching over program space, offering a general paradigm for scientific

agentsarxiv-cs-lg

PiDDM: Physics-Informed Differentiable Degradation Modeling for Lithium-Ion Battery State-of-Health Prediction

arXiv:2607.29095v1 Announce Type: new Abstract: Accurate prediction of lithium-ion battery state of health (SOH) is essential for reliable energy storage operation. However, purely data-driven models

RareSense: Rarity-Aware Similarity Search for Anomaly Retrieval in Transactional Data

arXiv:2607.28879v1 Announce Type: cross Abstract: Similarity search over sparse set-valued data is often dominated by frequent background attributes because classical measures such as Jaccard, cosine,

SAM+D: Parameter-Efficient Dimensional Lifting of SAM-Family Models via Depth-Routed LoRA and Depth Shifting

arXiv:2607.29033v1 Announce Type: new Abstract: Existing methods for adapting 2D foundation models such as SAM to 3D volumes either process slices independently---ignoring inter-slice context---or req

Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember

arXiv:2607.29468v1 Announce Type: new Abstract: Self-play agents can generate training problems without questions from target benchmarks, but their curricula lack persistent state: failures affect gra

safetyarxiv-cs-ai

Hardware

Studying quantization trade-offs for efficient inference deployment in machine translation

arXiv:2607.29397v1 Announce Type: new Abstract: Deploying large language models in realistic server environments poses challenges, as the system needs to provide high-quality responses with low latenc

hardwarearxiv-cs-cl

TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter

arXiv:2607.28657v1 Announce Type: new Abstract: Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential, which can be a barrier for non-expert users. This w

Yet another paper argues that LLMs aren’t close to doing real discovery.

Yet another paper argues that LLMs aren’t close to doing real discovery. MIT and Harvard argue LLMs are nowhere near doing real scientific discovery. They published a paper called “Evaluating Large La

safetygary-marcus--x

2 Aug 2026

A Graph-Native Bitemporal Memory Store for Conversational AI Agents

arXiv:2607.26520v1 Announce Type: cross Abstract: Conversational AI agents commonly lack persistent memory across sessions. The obvious fixes like injecting full chat histories into the context window

Chem World: A Large-Scale Benchmark and Physics-Informed Framework for Trustworthy Chemical Property Prediction

model-releasesarxiv-cs-lg

arXiv:2607.28079v1 Announce Type: new Abstract: Chemical property prediction plays a critical role in accelerating scientific discovery in chemistry, materials science, and drug development. However,

Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study

arXiv:2607.27577v1 Announce Type: cross Abstract: Heterogeneous recommendation feeds present complex challenges that extend beyond those found in highly homogeneous environments (e.g., music-only or v

safetyarxiv-cs-lg

IFHierBench: Hierarchical Instruction Following for Large Language Models

arXiv:2607.27912v1 Announce Type: cross Abstract: Instruction-following ability is critical for deploying large language models in real-world applications, where downstream components depend on the ou

Measuring Alignment With Reader Highlights Net of Position and Length

arXiv:2607.27739v1 Announce Type: cross Abstract: Context compression discards most of a document before a language model reads it, and is normally evaluated by downstream task accuracy - which makes

Model-Driven Requirements Configuration with Three-Valued Uncertainty Scoring

arXiv:2607.26220v1 Announce Type: cross Abstract: Context: Large Language Models (LLMs) offer natural-language flexibility for automated requirements elicitation but frequently generate structurally i

agentsarxiv-cs-ai

Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models

arXiv:2607.27304v1 Announce Type: cross Abstract: Medical vision-language models (VLMs) generate chain-of-thought (CoT) reasoning before answering clinical questions, but whether this reasoning causal

researcharxiv-cs-cv

Rethinking LLM-Judged Helpfulness as a Pedagogy Signal: A Pre-Registered Audit Across Tutor Models

arXiv:2607.28128v1 Announce Type: new Abstract: LLM tutoring poses a measurement problem: can a general-purpose helpfulness rubric distinguish direct answer-giving from pedagogical guidance? We audit

Scaling medical imaging report generation with multimodal reinforcement learning

arXiv:2601.17151v2 Announce Type: replace-cross Abstract: Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit ma

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

arXiv:2607.28269v1 Announce Type: new Abstract: The deployment of Vision-Language Models (VLMs) in critical domains like disaster management requires high-quality multimodal datasets, especially for t

researcharxiv-cs-cv

UrbanDS: A Graph-Guided LLM Multi-Agent System for Data-Intensive Urban Tasks

arXiv:2607.26724v1 Announce Type: new Abstract: Large language model (LLM) agents have been widely applied in automating data science tasks. However, existing methods typically rely on a limited set o

Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning

arXiv:2607.28478v1 Announce Type: new Abstract: As large language models (LLMs) continue to advance in complex reasoning tasks, they have learned to heavily prioritize explicit conditions provided in

Batten Down Your Packages: Mitigation Guidance for Supply Chain Compromise

model-releasesgoogle-cloud-ai

Written by: Kelli Vanderlee, Stuart Carrera For years, the cybersecurity industry's understanding of software supply chain compromise has been anchored by a few watershed events, including Russian cyb

BG-REAL: A Public Real-Data Anchored Benchmark for Background Manipulation Detection and Localization

arXiv:2607.26232v1 Announce Type: new Abstract: Background manipulation is a practical but under-specified image-forensics setting: the manipulated evidence can sit outside the salient foreground obje

Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility

model-releasesarxiv-cs-lg

arXiv:2607.26828v1 Announce Type: new Abstract: Large language models increasingly support scientific and algorithmic discovery through inference-time search over evaluated candidates. Existing adapti

Can AI agents conduct open-ended AI research? Early evidence from two case studies

arXiv:2607.27191v1 Announce Type: cross Abstract: Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is t

agentsarxiv-cs-lg

Credit Cards, Confusion, Computation, and Consequences: What Can We Uncover About Language Model Reasoning?

arXiv:2607.26952v1 Announce Type: new Abstract: We introduce CreditCardQA, the first financial literacy benchmark for numerical reasoning derived from real credit card agreements. The dataset contains

Entity Resolution in Practice: Lessons from a Self-Serve Pipeline

arXiv:2607.26298v1 Announce Type: new Abstract: We built and evaluated a self-serve entity resolution (ER) system on six benchmarks spanning 864 to 5M records, and three lessons emerged that are absen

Mental World Modeling

arXiv:2607.27201v1 Announce Type: new Abstract: World models enable a predictive substrate for planning and action, yet existing formulations merely answer a physical question: what/where it is, and h

agentsarxiv-cs-cl

one of the top cybersecurity models, post-trained from open weights by @depthfirstlabs on @FireworksAI_HQ long-horizon RL is as much an infr…

one of the top cybersecurity models, post-trained from open weights by @depthfirstlabs on @FireworksAI_HQ long-horizon RL is as much an infra problem as a research one: 100+ turn rollouts, async/pipel

safetyfireworks-ai--x

PRISM-Net: Patient-specific reference-guided inter-breast symmetry matching for three-class breast DCE-MRI classification

arXiv:2607.26799v1 Announce Type: new Abstract: Breast DCE-MRI AI is increasingly being explored for breast-level classification of no-lesion, benign, and malignant findings, beyond conventional lesio

researcharxiv-cs-cv

SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence

arXiv:2607.27066v1 Announce Type: new Abstract: Scientific figure assessment in peer review differs fundamentally from general image quality evaluation: a figure must be visually legible, faithfully s

safetyarxiv-cs-cv

Tutorials

SCOUT: Per-Context Reset Curricula for Sparse-Reward Reinforcement Learning

arXiv:2607.26417v1 Announce Type: new Abstract: Sparse-reward reinforcement learning often fails because rollouts from the unassisted evaluation start rarely reach later task stages. Reset curricula a

tutorialsarxiv-cs-lg

SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response

arXiv:2607.26791v1 Announce Type: cross Abstract: Large Language Model (LLM) agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces

Applications

See2Think: Do Multimodal Models Really Use Intermediate Visual States?

arXiv:2607.26769v1 Announce Type: new Abstract: Multimodal large language models increasingly use sketches, annotations, tools, and intermediate images during reasoning, but it remains unclear whether

applicationsarxiv-cs-cv

A Unified Benchmark and Modality-Adaptive Network for Day-and-Night Drone-View Geo-Localization

arXiv:2607.25778v1 Announce Type: new Abstract: Most existing drone-view geo-localization (DVGL) benchmarks contain drone imagery captured under a single illumination condition and lack geographically

Beyond Single-Episode Optimization: Sliding-Window Aware Generative Auto-Bidding for Long-Term Advertising Effectiveness

arXiv:2607.25233v1 Announce Type: new Abstract: Auto-bidding systems optimize bids to maximize value under efficiency constraints such as Cost-Per-Action (CPA). Existing methods treat each day as an i

Applications

Cognivia: A Cognitive Behavioral Therapy Copilot for Evidence-Based Mental Healthcare

arXiv:2607.25681v1 Announce Type: new Abstract: Cognitive distortion amplifies negative emotions and contributes to mental health disorders. Cognitive Behavioral Therapy (CBT) is an effective way to a

applicationsarxiv-cs-ai

CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization

arXiv:2607.25659v1 Announce Type: new Abstract: Rubric-based reinforcement learning enriches language model training by evaluating model outputs against explicit criteria. Yet in GRPO-style pipelines,

safetyarxiv-cs-ai

FinAbstain: Uncertainty-Calibrated Multimodal RAG for Selective Financial Forecasting

arXiv:2607.24875v1 Announce Type: new Abstract: Large language models (LLMs) can synthesize financial narratives but may express high confidence when evidence is sparse, stale, or contradictory. This

GAUGE: Grading Agent-Built Financial Models Without a Golden Answer

arXiv:2607.24889v1 Announce Type: cross Abstract: Financial models combine public disclosures with analyst assumptions to produce forecasts and valuations. While some components can be checked mechani

Latent Stability Analysis of Malware Representations Under Feature-Space Perturbations

arXiv:2607.24896v1 Announce Type: cross Abstract: Static malware detectors are commonly evaluated using clean-sample metrics such as accuracy, F1, ROC AUC, and PR AUC. However, these metrics provide l

researcharxiv-cs-ai

Local Ai

MARS: Multi-Agent Re-ranking for Repeat-Order Food Delivery Recommendation

arXiv:2607.25420v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in recommender systems, but it is often unclear how much performance can be obtained from strong pr

local-aiarxiv-cs-ai