Search: “evaluation”

PhysElite: How Far Are LLMs from Solving Olympiad-Level Physics Problems?

arXiv:2608.25097v2 Announce Type: replace Abstract: Understanding how (multimodal) large language models perform on physics problems requires benchmarks that reflect the difficulty and breadth of expe

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

DGX agent

arXiv:2609.30657v1 Announce Type: cross Abstract: Large language model email assistants are particularly vulnerable to indirect prompt injection because untrusted email content can be retrieved into t

safetyarxiv-cs-cl

28 Sept 2026

Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity

arXiv:2609.31505v1 Announce Type: new Abstract: Despite the growing capabilities of large language models (LLMs), prompt design remains largely heuristic and ad hoc. This project will explore extit{pr

Proper Scoring Rules for Right-Censored Survival Data

arXiv:2606.06393v2 Announce Type: replace Abstract: Proper scoring rules provide a rigorous theoretical basis for the training and evaluation of probabilistic forecasts. In survival analysis, such for

researcharxiv-cs-lg

PTC-Decoder: Towards Intelligent SLMs on Offline Resource-Constrained Edge Devices

arXiv:2609.30836v1 Announce Type: new Abstract: Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge:

RupeeBias: Auditing Demographic Bias in Indian Economic Guidance from Large Language Models

model-releasesarxiv-cs-cl

arXiv:2609.31245v1 Announce Type: new Abstract: Individuals turn to large language models (LLMs) for guidance across a wide range of economic tasks, from comparing loan options and planning savings to

SPADE: Escaping the Popularity-Similarity Frontier to Measure Serendipitous Recommendations

arXiv:2609.31164v1 Announce Type: cross Abstract: Recommender systems engineer serendipity to foster active exploration and break predictable consumption cycles. The problem with existing offline beyo

Applications

TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

arXiv:2609.30969v1 Announce Type: new Abstract: Tactile information is essential for contact-rich manipulation tasks in robotics. Vision-based tactile sensors make it particularly easy to design end-t

applicationsarxiv-cs-ro

TemplateCraft: Agentic Visual Template Generation

arXiv:2609.31451v1 Announce Type: cross Abstract: The growing popularity of short videos has driven demand for one-click content creation. Visual templates turn uploaded images into personalized conte

Safety

TISD: On-Policy Self-Distillation with Trajectory Intervention

arXiv:2609.30878v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) provides dense teacher targets, but evaluates them only along student-sampled rollouts. When the privileged teacher f

safetyarxiv-cs-ai

Uncertainty-Aware Federated Learning for Infant Movement Analysis

arXiv:2609.31463v1 Announce Type: cross Abstract: Infant movement analysis provides valuable biomarkers for the early identification of neurodevelopmental disorders. Recent advances in deep learning h

Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents

arXiv:2609.31076v1 Announce Type: new Abstract: Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agent

Where Compute Matters: Heterogeneous Attention for Efficient Video Diffusion

arXiv:2609.31050v1 Announce Type: new Abstract: Efficient video generation requires reducing the quadratic cost of self-attention over long spatio-temporal token sequences. Existing efficient-attentio

Applications

Worst thing about this is we have no idea if GPT-6 was specifically post trained on Nethack or not. Zero transparency about the training dat…

applicationsethan-mollick--x

Worst thing about this is we have no idea if GPT-6 was specifically post trained on Nethack or not. Zero transparency about the training data and environments. You can't independently evaluate the sig

26 Sept 2026

A Field-Deployable GNSS-based Navigation Stack for Outdoor Mobile Robots

arXiv:2609.28933v1 Announce Type: new Abstract: Outdoor robots require more than an accurate receiver and a path-tracking law: the navigation system must preserve geometric consistency from geographic

local-aiarxiv-cs-ro

Safety

Accelerating Video Diffusion via Training-Free Trajectory Routing

arXiv:2609.30096v1 Announce Type: cross Abstract: Video diffusion is computationally expensive, as it requires executing a large model across many denoising steps. Even with step-distillation, inferen

safetyarxiv-cs-ai

After the Hugging Face incident, we committed to conducting a much broader review of actions taken by our models during training and evaluat…

After the Hugging Face incident, we committed to conducting a much broader review of actions taken by our models during training and evaluation and to being transparent about our findings. This is an

model-releasesopenai--x

An Explainable DistilBERT-BiLSTM-Attention Framework for Binary and Multi-Class Hate Speech Detection

arXiv:2609.28703v1 Announce Type: cross Abstract: Hate speech on social media poses serious risks to social harmony, mental well-being, and public safety, making its timely and accurate detection esse

Benchmarking Argumentative Behaviour of LLMs: A Study of Defences Against Character Attacks

model-releasesarxiv-cs-cl

arXiv:2609.28673v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as argumentative agents in persuasive dialogues, necessitating rigorous evaluation of their debat

CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models

arXiv:2609.28813v1 Announce Type: new Abstract: Cinematography, the craft of visual storytelling through framing, lighting, and camera operation, fundamentally shapes how audiences perceive and emotio

Comparing YOLOv11 and YOLOv8 for instance segmentation of occluded and non-occluded immature green fruits in complex orchard environment

arXiv:2410.19869v4 Announce Type: replace Abstract: This study conducted a comprehensive performance evaluation on YOLO11 (or YOLOv11) and YOLOv8, the latest in the 'You Only Look Once' (YOLO) series,

researcharxiv-cs-cv

Decoding Imagined Speech: A Strictly Subject-Independent Approach Using EEG

arXiv:2609.29820v1 Announce Type: new Abstract: Imagined speech decoding from electroencephalography (EEG) has gained increasing attention as a potential communication pathway for individuals with sev

Encoded but Not Decoded: Layer-Localized Evidence for a Three-Level Gap in LLM Syntax

model-releasesarxiv-cs-cl

arXiv:2609.29848v1 Announce Type: new Abstract: A language model can fail a syntactic test in two distinct ways: by not encoding the relevant structure, or by encoding it but failing to use it at the

Agents

EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discovery

arXiv:2609.29016v1 Announce Type: cross Abstract: AI-driven scientific discovery accelerates research by autonomously developing solutions and designs. Large language model (LLM) agents support this p

agentsarxiv-cs-lg

ExplorationBench: Measuring AI Systems' Exploration in Verifiable Alien Worlds

arXiv:2609.30199v1 Announce Type: new Abstract: Scientific discovery begins where known problems end. There, AI systems must engage in exploration: framing hypotheses, designing experiments, and itera

fable.intermittent: benchmarking probabilistic forecasting methods for intermittent time series

model-releasesarxiv-cs-lg

arXiv:2609.28607v1 Announce Type: new Abstract: Intermittent time series are common in spare-parts demand and retail sales. Since the cost of forecast errors is typically asymmetric, decisions such as

FlowAtom: Atom-Based Evidence Aggregation for Multi-Label Website Fingerprinting

arXiv:2609.29330v1 Announce Type: new Abstract: Identifying the set of monitored websites in mixed encrypted traffic is challenging because an individual flow often provides only partial evidence of w

researcharxiv-cs-lg

Tutorials

Generative Atmospheric Super-Resolution from Heterogeneous In Situ Observations through Composable Interfaces

arXiv:2609.29027v1 Announce Type: new Abstract: Atmospheric observations are sparse, heterogeneous, and unevenly distributed, whereas many generative atmospheric models learn distributions over regula

tutorialsarxiv-cs-lg

Image Fidelity is Not Field Fidelity: Joint Thermodynamic Reconstruction and Error Localization in Neural Tomography

arXiv:2609.28868v1 Announce Type: new Abstract: Neural fields for scientific tomography are optimized from 2D images, but the actual quantity of interest is often a latent 3D physical field. Because t

local-aiarxiv-cs-lg

Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER

model-releasesarxiv-cs-lg

arXiv:2609.29101v1 Announce Type: new Abstract: Medical Named Entity Recognition (NER) for low-resource languages remains a challenging task due to high linguistic variability and a scarcity of domain

Likelihood Ranking doesn't Scale Like Prompting in LLMs

arXiv:2609.29390v1 Announce Type: new Abstract: LLM evaluation is commonly performed either by prompting models to produce answers or by scoring candidate outputs with likelihood-based metrics. In mul

researcharxiv-cs-cl

MeshHeal: Two-Timescale Self-Healing for Gray Failures in Decentralized LLM Agent Networks

arXiv:2609.29015v1 Announce Type: new Abstract: Decentralized LLM-based multi-agent systems coordinate through local interactions, but an agent can remain responsive while its task-solving quality per

Applications

NarrateAI: production-ready LLM quality assurance on Amazon Bedrock

NarrateAI delivers production-ready LLM quality assurance on Amazon Bedrock. This post details five techniques—adaptive pipeline orchestration, cross-account multi-model failover, real-time streaming

applicationsaws-ml-blog

Not All Confusion Is Equal: A Source-Aware Uncertainty Diagnosis for Fine-Grained Aircraft Detection

arXiv:2609.29959v1 Announce Type: new Abstract: Fine-grained object detectors are commonly evaluated with confusion matrices, which show where the model is confused but not why, nor whether the confus

Operator Packages, Proposer Strength, and Construction-Family Plateaus in Office-Scale Verified Search

arXiv:2609.29636v1 Announce Type: cross Abstract: Verified search, in which a language model proposes programs, a hard evaluator scores them, and selection keeps the best, has recently moved mathemati

PUBG Ally: A Conversational Embodied Agent as an AI Teammate

arXiv:2609.29837v1 Announce Type: new Abstract: We introduce PUBG Ally, an embodied agent for PUBG: BATTLEGROUNDS that can reason, act autonomously, and play alongside players as a voice-enabled teamm

SkinAgent AI: A Safety-Grounded Multimodal Agentic Framework for Non-Diagnostic Skincare Support

arXiv:2609.29341v1 Announce Type: new Abstract: Consumer-facing skincare AI must coordinate visual evidence, product information, tool use, and user-facing actions within explicit evidence and safety

SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questions

arXiv:2512.14277v2 Announce Type: replace-cross Abstract: The advent of large language models is contributing to the emergence of novel approaches that promise to better tackle the challenge of genera

Spectral-Guided Diffusion: Accelerating Inference via Static Spectral Layer Scheduling

arXiv:2609.29505v1 Announce Type: new Abstract: Diffusion inference repeatedly evaluates the same large network. We ask whether pretrained weights alone can identify residual branches that need not be

researcharxiv-cs-lg

Agents

Stochastic Semantic Evidence Graphs: Uncertainty Propagation and Governance for Agentic AI

arXiv:2609.29703v1 Announce Type: new Abstract: AI-agent evaluations usually inspect a final answer, yet error may enter through evidence, retrieval, prompting, generation or decision mapping. We intr

agentsarxiv-cs-cl

Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models

arXiv:2609.30048v1 Announce Type: new Abstract: If a language model can recognize code it wrote, it may favor that code as a judge, and instances of one model monitoring each other could collude. We t

The Tokens Remember: When Tokenization Bypasses Knowledge Editing and Unlearning

arXiv:2609.29045v1 Announce Type: cross Abstract: Open-weight LLMs give downstream users control over the inference stack, but this flexibility can undermine post-release guarantees that sensitive kno

Towards Deployable Underwater Vessel Classification

model-releasesarxiv-cs-lg

arXiv:2609.29179v1 Announce Type: cross Abstract: We propose a compact underwater acoustic classification framework combining multi-representation feature engineering, temporal statistical pooling, an

Wearable ECG Quality Assessment: A Deep Learning and Ambulatory Context-Awareness Approach

arXiv:2609.29396v1 Announce Type: new Abstract: This paper presents and evaluates a Deep Learning-based (DL-based) Signal Quality Assessment (SQA) model to distinguish between clean and noisy ambulato

A new, no-compromises database architecture for the agentic era

model-releasesgoogle-cloud-ai

Entire database engineering careers have been spent on a single question: How do you scale an OLTP workload without compromising the system of record that owns the data? Exadata answered the question

A Shared Encoder Is Not a Shared Task: Conditional Comparison for Deep Expert Pools

arXiv:2609.27866v1 Announce Type: cross Abstract: Sharing a deep encoder does not, by itself, fix the central confound of task-comparison scores. We show that cross-evaluated heads on a frozen shared

AgroBench: A Reproducible Multimodal Benchmark for Weakly Supervised Crop Yield Learning from County Statistics and Pixel Observations

arXiv:2609.26809v1 Announce Type: new Abstract: Reliable agricultural yield statistics are typically reported at coarse administrative scales, whereas modern geospatial machine learning methods requir

Beyond Poetry: Can Large Language Models Generate Classical Arabic Maqamat?

arXiv:2609.28245v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong performance in creative text generation, yet their ability to produce culturally grounded and stylistic