Search: “evaluation”

On the Recall Scaling Laws in Mamba: A Theoretical and Mechanistic Study via Hashing

arXiv:2609.07681v1 Announce Type: new Abstract: Associative Recall (AR) is the cognitive ability to learn and retrieve links between items in memory. In NLP, AR is used as a benchmark for evaluating t

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

DGX agent

arXiv:2609.07398v1 Announce Type: new Abstract: World-Action Models inherit world knowledge from video-generative priors, and channel it into executable control signals through embodied experience. Ex

applicationsarxiv-cs-ro

9 Sept 2026

SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?

arXiv:2609.09113v1 Announce Type: cross Abstract: While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a mis

SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code

arXiv:2609.08155v1 Announce Type: new Abstract: Scientific figures are the interface through which research claims are inspected and reused, but final published panels rarely expose the data or plotti

SynthRCT: Scalable Conditional Deformation Synthesis for Synthetic Repeat CT Generation

arXiv:2609.08627v1 Announce Type: new Abstract: In proton therapy, plans are typically optimized on a single planning CT, making robustness evaluation essential under anatomical changes. However, curr

researcharxiv-cs-cv

The Art of Hierarchical Competing Patterns: Gaussian Process Optimization of Hyphenation

arXiv:2609.07638v1 Announce Type: cross Abstract: Hyphenation patterns remain a compact and widely deployed solution for word breaking in typesetting systems, text processors, and web rendering engine

The Oversight Gap: What LLM Safety Monitors Miss, and Why It Is Not Capability

arXiv:2609.07162v1 Announce Type: new Abstract: Several properties safety monitors are asked to certify, among them cross-tenant noninterference, sandbagging and evaluation awareness, are 2-safety hyp

Tracking-by-detection in Multi-object Tracking: Survey and Experiments

arXiv:2609.08265v1 Announce Type: new Abstract: Multi-object tracking (MOT) is an essential computer vision task that simultaneously tracks multiple objects in video sequences, with various applicatio

agentsarxiv-cs-cv

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

arXiv:2609.08040v1 Announce Type: cross Abstract: The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing de

VI-Bench: Benchmarking Prompt Inversion from AIGC Videos

arXiv:2609.08079v1 Announce Type: new Abstract: Recent advances in video generation have made prompt-based control increasingly central to AIGC video generation. Prompts specify what a video should de

What Are You Doing? A Closer Look at Controllable Human Video Generation

arXiv:2503.04666v2 Announce Type: replace Abstract: High-quality benchmarks are crucial for driving progress in machine learning research. However, despite the growing interest in video generation, th

Generating Bad Apple autonomously from a single initial state using a tiny recurrent dynamical system (417k params) [P]

model-releasesr-machinelearning

A few weeks ago, I saw this post where the author trained a SIREN MLP to implicitly memorize Bad Apple as a coordinate function: (t, y, x) to pixel. That got me curious about a slightly different form

8 Sept 2026

Tutorials

Govern models with MLflow and Amazon SageMaker AI Model Registry sync: Part 1

Managed MLflow on Amazon SageMaker AI now syncs richer model metadata (training metrics, evaluation results, inference specs, and lineage) into the SageMaker AI Model Registry, with lifecycle stage pr

tutorialsaws-ml-blog

8 Sept 2026

I reduced image-processing token usage by ~95% compared with GPT-4o direct vision, while maintaining roughly the same accuracy.How significant is that?[P]

model-releasesr-machinelearning

I'm testing a new approach for reducing the cost of image-based LLM inference. I evaluated it on the MOMA Graph benchmark, using 1,315 questions. Compared with using GPT-4o to process the original ima

8 Sept 2026

Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal

arXiv:2602.11558v2 Announce Type: replace Abstract: Brain foundation models (BFMs) are advancing neurotechnology by learning transferable representations from neural signals, with broad potential in c

Candidate Comparability Before Promotion: Conditional Validation in Adaptive Network Intrusion Detection

arXiv:2609.04388v1 Announce Type: cross Abstract: Adaptive network intrusion detection systems retrain classifiers after drift alarms, but an alarm detects change; it does not establish that a challen

Safety

CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving

arXiv:2609.04807v1 Announce Type: cross Abstract: Multi-vehicle cooperative autonomous driving enhances the safety and reliability of autonomous driving systems through information sharing among conne

safetyarxiv-cs-cv

Hardware

Compact Neural Appearance Models for Efficient Gaussian Splatting

arXiv:2609.05255v1 Announce Type: new Abstract: Explicit primitive-based radiance fields such as 3D Gaussian Splatting typically model view-dependent appearance using low-order spherical harmonics (SH

hardwarearxiv-cs-cv

Hardware

Compute-in-Memory Attention: A Time-Domain Analog Softmax Circuit with RC-Tunable Temperature

arXiv:2609.04266v1 Announce Type: cross Abstract: Softmax is a key operation in Transformer attention, but its exponentiation and normalization add significant overhead in compute-in-memory (CIM) acce

hardwarearxiv-cs-lg

Safety

Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning

arXiv:2609.04485v1 Announce Type: cross Abstract: We evaluate three open-weight LLMs (Gemma3-12B from the USA, Bielik-11B-v3 from Poland, and Qwen3-4B from China) against World Values Survey Wave 7 da

safetyarxiv-cs-ai

Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory

arXiv:2609.05245v1 Announce Type: new Abstract: Human knowledge is inherently structured and interdependent: mastery of a concept requires prior mastery of its prerequisites, a principle formalized by

researcharxiv-cs-ai

From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making

arXiv:2609.05149v1 Announce Type: new Abstract: Vision-Language Models are commonly evaluated through their final predictions, but understanding whether these decisions are grounded in visual evidence

GLOW: Graph-Language Co-Encoding for Agentic Workflow Performance Prediction

arXiv:2512.15751v2 Announce Type: replace-cross Abstract: Agentic Workflows (AWs) have emerged as a promising paradigm for solving complex tasks. However, automatically generating high-quality AWs rem

LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models

model-releasesarxiv-cs-ro

arXiv:2609.05178v1 Announce Type: new Abstract: Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA met

Applications

Long Horizon Transformer Quantile Fault Prediction for Multi Site Industrial Predictive Maintenance

arXiv:2609.04840v1 Announce Type: new Abstract: Long-horizon predictive maintenance requires models to distinguish slowly evolving degradation from normal operating-regime variation over planning wind

applicationsarxiv-cs-ai

Mitra-v2 Technical Report

arXiv:2609.04540v1 Announce Type: new Abstract: We introduce Mitra-v2, a tabular foundation model that delivers state-of-the-art performance on real-world classification and regression problems, from

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

arXiv:2609.04842v1 Announce Type: cross Abstract: Time-series data in clinical settings is crucial for capturing dynamic changes in a patient's health over time, enabling timely diagnosis, personalize

Molecular Deja Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

arXiv:2609.05381v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated on molecular property benchmarks, but accuracy cannot distinguish a model that predicts a proper

Nepali Passport Question Answering: A Low-Resource Dataset for Public Service Applications

arXiv:2603.13320v2 Announce Type: replace-cross Abstract: Nepali, a low-resource language, faces significant challenges in building an effective information retrieval system due to the unavailability

researcharxiv-cs-lg

Safety

On Epistemic Diversity in Large Language Models

arXiv:2609.04835v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used not only to retrieve information, but to answer questions, explain, teach, and support inquiry. In su

safetyarxiv-cs-cl

One Diffusion Model, Two Roles: Guided Trajectory Planning and Safety-Critical Scenario Generation in Closed-Loop Simulation

arXiv:2609.04921v1 Announce Type: cross Abstract: Diffusion probabilistic models can capture the multi-modal, interaction-rich distribution of joint future trajectories in driving scenes. We show that

Local Ai

Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys

arXiv:2609.04382v1 Announce Type: cross Abstract: We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while leaving an observable channel u

local-aiarxiv-cs-lg

QoNext: Towards Next-generation QoE for Foundation Models

arXiv:2509.21889v3 Announce Type: replace Abstract: Existing evaluations of foundation models predominantly focus on output correctness, treating interaction as a static exchange of information. Howev

TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audio

model-releasesarxiv-cs-cl

arXiv:2609.04452v1 Announce Type: new Abstract: Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue re

Unifying ICL, SFT, KL-Regularized RL Through a Bayesian Lens

arXiv:2609.05111v1 Announce Type: new Abstract: Large language models are now trained and evaluated under a diverse set of paradigms: supervised fine-tuning (SFT), few-shot in-context learning (ICL),

VISTA: Dense Multi-Label Classroom Coding with Vision-Language Models

arXiv:2609.04550v1 Announce Type: new Abstract: Video-language benchmarks are usually constructed by the dataset authors without published reliability statistics, leaving the noise floor of the constr

You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments

arXiv:2609.04384v1 Announce Type: cross Abstract: Chinese online comments often convey social meaning through indirect and playful language that is hard to interpret without context. Existing evaluati

A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms

arXiv:2609.04170v1 Announce Type: new Abstract: Multi-agent AI science ecosystems rely on agents possessing tools that allow them to communicate, coordinate, and build on each other's work. Yet this s

A Spectral Phase Admissibility Certificate for Complex Linear Maps

arXiv:2609.02911v1 Announce Type: cross Abstract: The paper imports the Kontsevich Segal Witten criterion from quantum gravity into machine learning to evaluate complex linear maps Standard techniques

Anonymization, Not Elimination: Utility-Preserved Speech Anonymization

arXiv:2604.17000v1 Announce Type: cross Abstract: The growing reliance on large-scale speech data has made privacy protection a critical concern. However, existing anonymization approaches often degra

researcharxiv-cs-ai

Causal-Counterfactual RAG: The Integration of Causal-Counterfactual Reasoning into RAG

arXiv:2509.14435v3 Announce Type: replace Abstract: Large language models (LLMs) have transformed natural language processing (NLP), enabling diverse applications by integrating large-scale pre-traine

Counterexamples as Feedback for Agent Self-Correction

arXiv:2609.02892v1 Announce Type: cross Abstract: Single-turn code-generation metrics understate a central property of deployed agents: whether they can repair a wrong artifact after receiving concret

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

arXiv:2609.03526v1 Announce Type: new Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cult

Efficient Test-Time Adaptation through Human-AI Interaction

arXiv:2609.04141v1 Announce Type: new Abstract: AI agents are trained on population-scale data to encode broad capabilities spanning those of many practitioners. Yet the artifacts they produce rarely

Local Ai

Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailable

arXiv:2609.04127v1 Announce Type: new Abstract: Large language models are increasingly used to support organizational decisions, yet users often lack a principled basis for assessing whether to rely o

local-aiarxiv-cs-ai

Evolving Excellence: Automated Optimization of LLM-based Agents

arXiv:2512.09108v2 Announce Type: replace-cross Abstract: Agentic AI systems built on large language models (LLMs) offer significant potential for automating complex workflows, from software developme

Explainable Convolutional Neural Networks for Retinal Fundus Classification and Cutting-Edge Segmentation Models for Retinal Blood Vessels from Fundus Images

arXiv:2405.07338v2 Announce Type: replace-cross Abstract: Early detection of vision-threatening conditions such as diabetic retinopathy, glaucoma, and age-related macular degeneration depends on retin

researcharxiv-cs-cv

FiMI Banking: A Sovereign Model for Indian Retail Banking

arXiv:2609.03960v1 Announce Type: new Abstract: Banks need conversational systems that can answer product questions, assist customers with account-related requests, and operate safely within strict op