Search: “evaluation”

MBA: Multimodal Benchmark and Agents for Real-World Business Ideation

arXiv:2608.11616v1 Announce Type: new Abstract: Agentic systems powered by large language models (LLMs) have opened new opportunities for business ideation. Yet existing approaches remain confined to

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning

DGX agent

arXiv:2608.11408v1 Announce Type: new Abstract: Prior white-box studies show that large language models can retain latent traces of target knowledge after unlearning, even when the knowledge is no lon

researcharxiv-cs-cl

13 Aug 2026

Monitor on-premises and multi-cloud AI agents with AgentCore Observability

Set up Amazon Bedrock AgentCore Observability for AI agents running outside AWS: on-premises, on GCP, on Azure, or on developer machines. This walkthrough uses the AWS Distro for OpenTelemetry (ADOT)

tutorialsaws-ml-blog

Preference Tree Optimization: Enhancing Goal-Oriented Dialogue with Look-Ahead Simulations

arXiv:2608.12062v1 Announce Type: cross Abstract: Developing dialogue systems capable of engaging in multi-turn, goal-oriented conversations remains a significant challenge, especially in specialized

Program Semantic Inequivalence Game with Large Language Models

arXiv:2505.03818v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) can achieve strong performance on everyday coding tasks, but they can fail on complex tasks that require non-triv

RelShap: Relationally Consistent Shapley Explanations

arXiv:2608.11508v1 Announce Type: new Abstract: Machine learning pipelines commonly flatten relational data into single-table representations, discarding structural constraints. Widely used Shapley va

Robustness of AI-Art Detectors under Generator Shift

arXiv:2608.11643v1 Announce Type: new Abstract: Text-to-image generative models have advanced rapidly, with modern Diffusion Transformer architectures producing images that are increasingly difficult

SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks

arXiv:2608.12127v1 Announce Type: new Abstract: Model routing aims to select the most suitable model from a candidate pool for each query, balancing quality and cost. Existing VLM routing research is

SportD: How do VLMs physically strategize?

arXiv:2607.14616v3 Announce Type: replace Abstract: Vision-language models (VLMs) can describe a scene, but can they act well within one? We study whether VLMs can make sound strategic decisions, usin

The Next Challenge for Agentic Cybersecurity: A Realistic, Contamination-Free Reverse Engineering Benchmark

arXiv:2608.11469v1 Announce Type: cross Abstract: AI agents are rapidly improving in cybersecurity capabilities when the source code is available for analysis, yet much of the software most consequent

The Sleeping Agent: What Gist-Based Context Compression Loses and Why

arXiv:2608.11775v1 Announce Type: new Abstract: Gist-based context compression---summarising older conversation history into compact representations---is a common approach in long-horizon language mod

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

arXiv:2608.11415v1 Announce Type: cross Abstract: Large language models are being proposed as agents in scientific workflows, in domains where no downstream verifier exists. Such deployment assumes th

UrgenT Help Detecting Performance Regressions Using Machine Learning and Hardware Counters [P]

tutorialsr-machinelearning

I’m working on performance regression detection using machine learning/anomaly detection. My setup is basically: Healthy runs are used to learn normal behaviour Regression runs are used to see whether

WLC: Weber-Inspired Local Contrast Metric for Low-Altitude Image Fusion

arXiv:2512.15211v4 Announce Type: replace Abstract: Infrared and visible image fusion is a pivotal technology in low-altitude Unmanned Aerial Vehicle (UAV) reconnaissance missions, enabling robust tar

A Convolutional Layer Activation Dimensionality Reduction for Out-of-Distribution and Adversarial Attack Detection Methods

arXiv:2608.10203v1 Announce Type: new Abstract: Despite the success of convolutional neural networks in image classification tasks and their general application in multi-modal models, their susceptibi

safetyarxiv-cs-cv

Applications

A Cost-Efficient Routing Pipeline for Multilingual Short-Text Classification Using Small Language Models

arXiv:2608.10939v1 Announce Type: cross Abstract: Multilingual short-text classification supports operational systems such as content moderation, customer support routing, and intent recognition, yet

applicationsarxiv-cs-ai

A Dataset and Benchmark for Optical Music Recognition of String Quartet Scores

arXiv:2608.10978v1 Announce Type: new Abstract: Optical music recognition (OMR) transcribes music scores into digital formats. While the field has advanced significantly on monophonic and piano-form s

Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique

arXiv:2608.10430v1 Announce Type: cross Abstract: Large Language Models (LLMs) deployed as AI agents frequently exhibit user specification-grounding failures, executing hallucinated, undesired actions

Agentic Instruction Data Selection: Let DataMaster Interpret Your Intent

arXiv:2608.10579v1 Announce Type: new Abstract: Although existing instruction data selection methods have introduced various metrics, the inherent complexity of real-world datasets makes it impractica

Applications

Assessing Reliability of BERT-Based Models on Question Answering Tasks

arXiv:2608.10806v1 Announce Type: new Abstract: Reliability estimation of large language models is in many cases as crucial as their accuracy, as reliable models are more trustworthy, robust, and suit

applicationsarxiv-cs-cl

Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting

model-releasesarxiv-cs-lg

arXiv:2608.10891v1 Announce Type: new Abstract: Time series forecasting in privacy-sensitive domains often requires training models on released data rather than original observations. Synthetic time s

Beyond Detection Accuracy: Measuring Explanation Cost, Stability, and Utility for Resource-Aware IoT Intrusion Detection

arXiv:2608.10349v1 Announce Type: cross Abstract: Machine-learning intrusion-detection studies commonly emphasize predictive accuracy while treating explanation generation as a computationally free po

Beyond Forecasting: Recasting Volatility Control as a Routing Problem

arXiv:2608.10375v1 Announce Type: cross Abstract: Volatility control converts risk estimates into portfolio exposure, yet existing approaches often rely on a fixed volatility estimator or a pre-define

safetyarxiv-cs-ai

BreastMammo and DenseMammo: Benchmarks for Mammography Domain Generalization

arXiv:2608.10271v1 Announce Type: cross Abstract: Breast density classification is a critical component of breast cancer risk assessment, yet AI models often struggle to generalize across clinical sit

Calibrating Post-Training Feature Shifts for LLM Data Contamination Detection

arXiv:2608.10462v1 Announce Type: new Abstract: Large language models (LLMs) are trained on massive and largely undisclosed corpora that may contain copyrighted or privacy-sensitive content. Data cont

researcharxiv-cs-cl

Can Computational Reducibility Lead to Transferable Models for Graph Combinatorial Optimization?

arXiv:2603.02462v2 Announce Type: replace-cross Abstract: A key challenge in developing unified neural solvers for combinatorial optimization (CO) is the efficient generalization of models from a give

Hardware

CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening

arXiv:2608.10506v1 Announce Type: cross Abstract: Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resourc

hardwarearxiv-cs-lg

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

arXiv:2608.11150v1 Announce Type: new Abstract: While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle

Chartography: A Benchmark for Professional Chart Understanding

arXiv:2608.10677v1 Announce Type: new Abstract: Professionals across medicine, engineering, finance, manufacturing, and the sciences often make consequential decisions from charts. Existing chart benc

ComBodied Agents: a New Paradigm of Human-Centric Agentic AI

arXiv:2608.10915v1 Announce Type: new Abstract: After an older adult misses a medication dose, a software agent can send another reminder and an embodied agent can bring the medication. Yet neither ex

Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance

arXiv:2608.10434v1 Announce Type: new Abstract: Machine learning-based Intrusion Detection Systems (IDS) have demonstrated superior performance in securing Unmanned Aerial Vehicle (UAV) networks. Howe

DEFT: Data-Efficient Frequency-domain Top-k Sampling via Inverse Discrete Fourier Transform for Spatiotemporal Dynamical Systems Modeling

arXiv:2608.11019v1 Announce Type: new Abstract: Modeling spatiotemporal dynamical systems governed by partial differential equations (PDEs) poses two major challenges: it either requires expensive phy

DoseBridge: Denoising Diffusion Bridge Model for Dose Prediction in Lung Intensity-Modulated Proton Therapy

arXiv:2608.10173v1 Announce Type: new Abstract: Most radiotherapy dose-prediction models use only CT images and anatomical structures, although intensity-modulated proton therapy (IMPT) dose also depe

Energy and Performance Benchmarking of Deep Learning Models for Breast Cancer Detection

arXiv:2608.09996v1 Announce Type: cross Abstract: Recent advances in machine learning have greatly improved breast cancer detection, enabling more accurate and timely diagnosis. Deep learning (DL) mod

Generation-Step-Aware Framework for Cross-Modal Representation and Control in Multilingual Speech-Text Models

arXiv:2601.17387v3 Announce Type: replace Abstract: Multilingual speech-text models rely on cross-modal language alignment to transfer knowledge between speech and text, but it remains unclear whether

safetyarxiv-cs-cl

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

model-releasesarxiv-cs-lg

arXiv:2509.13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. Whi

HUI360: A 360{eg} Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation

arXiv:2608.11051v1 Announce Type: new Abstract: As robots increasingly operate in human-populated environments, anticipating human intentions is essential for enabling proactive and socially aware beh

Human versus Computer Vision

arXiv:2608.10181v1 Announce Type: new Abstract: Computer vision saliency models predict where people will look, one map per image, and a billion-dollar predicted-attention industry sells those maps in

tutorialsarxiv-cs-cv

HyperShape: Hyperelasticity Across Diverse Shapes

model-releasesarxiv-cs-lg

arXiv:2608.09938v1 Announce Type: cross Abstract: Hyperelastic deformations are highly sensitive to domain geometry and boundary conditions, making generalization across both a critical capability for

Hypothesis Frontier: Verifier Guided LLM and Symbolic Search for First-Order Induction

arXiv:2608.10843v1 Announce Type: new Abstract: First-order concept synthesis asks a system to infer one formula that classifies labeled objects consistently across several finite relational structure

tutorialsarxiv-cs-ai

Implicit representations are dead. Long live explicit primitives!

arXiv:2608.10001v1 Announce Type: cross Abstract: Continuous parameterization of medical data has emerged as a powerful paradigm for resolution-independent image representation. While Implicit Neural

IO Factory: Simulating AI-Enabled Influence Campaigns at Scale

arXiv:2608.10920v1 Announce Type: new Abstract: We introduce IO Factory, an AI-driven framework for simulating information and influence campaigns as fully integrated, traceable processes. The threat

safetyarxiv-cs-ai

Is There Really a Camouflaged Object? Towards Realistic Camouflaged Object Detection

arXiv:2608.11135v1 Announce Type: new Abstract: Camouflaged object detection (COD) aims to segment objects that are visually concealed in their surroundings and has attracted increasing attention in r

Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy

arXiv:2608.09992v1 Announce Type: cross Abstract: Controllable generation guided by external knowledge is a key requirement in modern generative deep learning applications, enabling the synthesis of s

MD-ProTector: Positioning Multiple Data-Driven Prototypes for LLM-Generated Text Detection

arXiv:2608.10459v1 Announce Type: cross Abstract: As LLM-generated content becomes more sophisticated, detection systems for distinguishing those texts from human-written text must operate at scale wh

Measure the Sim-to-Real Gap: Designing an Affordable Real-World Benchmark Platform for Reinforcement Learning in AIoT Systems

arXiv:2607.10309v2 Announce Type: replace Abstract: Reinforcement learning (RL) is commonly employed to enhance the performance of autonomous systems, including the Autonomous Internet of Things (AIoT

Measuring Semantic Abstractness of SAE Features via Nonlocality

arXiv:2608.10537v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) have helped uncover mechanistic explanations for LLM behaviours such as reasoning, jailbreaking etc., via understanding the c

MEGA: Self-Evolving Agent Optimization Infrastructure via Wisdom Graph

arXiv:2608.10504v1 Announce Type: new Abstract: As coding agents increasingly handle implementation, the central challenge shifts from building individual agents to building an infrastructure that sys