Search: “evaluation”

What actually makes an AI application good or not? Thanks to @StackOverflow for hosting @the_bunny_chen on the Stack Overflow Podcast (even though he is giving away all of our secrets!) Listen here: h

tutorialsfireworks-ai--x

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

What You See Is What You Get: Observation-Aligned Supervision for Chart-to-Code Generation

DGX agent

arXiv:2607.04726v1 Announce Type: new Abstract: Chart-to-code generation is commonly trained with supervised fine-tuning on reference plotting scripts, implicitly treating the gold code as a fully obs

researcharxiv-cs-cl

7 Jul 2026

When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games

arXiv:2607.05132v1 Announce Type: cross Abstract: As large language models are deployed as autonomous agents that communicate intentions before acting, a critical safety question is whether agents tha

safetyarxiv-cs-cl

When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents

arXiv:2607.05189v1 Announce Type: cross Abstract: Persistent personal agents combine long-term memory with access to users' external environments, enabling personalized foreground assistance and proac

When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

model-releasesarxiv-cs-lg

arXiv:2607.04919v1 Announce Type: new Abstract: Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We pro

When Does High-CFG Diffusion Inversion Fail? A Controlled Study of Prompt--Latent Interactions

model-releasesarxiv-cs-cv

arXiv:2607.04731v1 Announce Type: new Abstract: Text-guided diffusion inversion is central to image editing, where an image is mapped to an initial latent and then edited by replaying the denoising pr

When Does Small Data Work? Accuracy and Efficiency Trade-offs Between Tabular Foundation Models and Conventional Methods for Crowd-State Classification at Hajj and Umrah

arXiv:2607.04013v1 Announce Type: cross Abstract: Learning from few labeled examples is a central challenge in tabular machine learning, and it becomes the binding constraint in domains where labeling

When Geometry Aligns: Dihedral Hidden-State Transformations in UNet, ViT, and DiT Architectures

arXiv:2607.03580v1 Announce Type: cross Abstract: Diffusion architectures now encompass convolutional UNets as well as transformer-based designs such as Diffusion Transformers (DiTs), inspired by Visi

researcharxiv-cs-cv

When is a System Discoverable from Data? Discovery Requires Chaos

arXiv:2511.08860v2 Announce Type: replace-cross Abstract: The deep learning revolution has spurred a rise in advances of using AI in sciences. Within physical sciences the main focus has been on disco

Local Ai

When Words Predict Workload

arXiv:2607.04951v1 Announce Type: cross Abstract: Standard distributed ac{llm} schedulers rely on static token counts or rolling latency averages, making them susceptible to failures on statutorily co

local-aiarxiv-cs-cl

Where do LLMs Fall Short in CBT-Guided Affective Reasoning?

arXiv:2607.02885v1 Announce Type: cross Abstract: Cognitive Behavioral Therapy (CBT) provides a structured framework for understanding a user's mental state by examining the interaction between cognit

Which Algorithm Specification Formats Help Language Models Implement Machine Learning Algorithms?

arXiv:2607.03158v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to implement algorithms from research manuscripts, but papers often leave implementation choices im

Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models

model-releasesarxiv-cs-cl

arXiv:2607.04640v1 Announce Type: new Abstract: We study how correctness is assembled inside aligned language models, not only whether the final answer is right. Using layer-wise difference-in-differe

And yes, costs matter. But (1) some of your tokens are basically R&D spending for yourself to figure out what these systems can do and (2) F…

applicationsethan-mollick--x

Ethan Mollick discusses how token costs for AI systems shouldn't be viewed purely as expenses, arguing that some token usage represents research and development spending to understand AI capabilities,

6 Jul 2026

'Ghost memory' is a real problem with agents. You might have seen the issue where a long-running agent still confidently repeats a user fact…

'Ghost memory' is a real problem with agents. You might have seen the issue where a long-running agent still confidently repeats a user fact that stopped being true weeks ago? New research names the f

model-releasesdair-ai--x

6 Jul 2026

Shift into high gear with agents: Securing the software-defined vehicle

model-releasesgoogle-cloud-ai

The automotive industry is at a pivotal crossroads as it hits the gas on adopting new technology. The era of the traditional connected vehicle has shifted into the age of the software-defined vehicle

6 Jul 2026

introducing tinyrouter i reverse engineered the routing architecture behind Skana AI's Fugu and built replication for open frontier models. …

model-releasesclem-delangue--x

introducing tinyrouter i reverse engineered the routing architecture behind Skana AI's Fugu and built replication for open frontier models. it's a tiny ~10K parameter LLM router that learns which mode

4 Jul 2026

Agents

A Convex Obstacle Avoidance Formulation

arXiv:2512.13836v2 Announce Type: replace-cross Abstract: Autonomous driving requires reliable collision avoidance in dynamic environments. Nonlinear Model Predictive Controllers (NMPCs) are suitable

agentsarxiv-cs-ro

A Multi-Branch Hierarchy-Aware Framework for Heterogeneous Audio Classification

arXiv:2607.01974v1 Announce Type: cross Abstract: This technical report describes our system for Task 1 of the DCASE 2026 Challenge, which aims to classify heterogeneous audio recordings according to

A Reconfigurable Rocker-Bogie Robot for High Step Climbing and Turning

arXiv:2607.01554v1 Announce Type: new Abstract: This study proposes a reconfigurable rocker-bogie mechanism that achieves efficient turning motion with a small number of actuators while maintaining hi

researcharxiv-cs-ro

A Stereo Visual SLAM System Using Object-Level Motion Estimation and Geometric Filtering Based on Cross Disparity

arXiv:2607.02005v1 Announce Type: new Abstract: This paper presents OCD SLAM, a dynamic stereo visual SLAM framework that extends ORB-SLAM2 by jointly addressing dynamic objects and dynamic features i

researcharxiv-cs-ro

Tutorials

A Survey of Circuit Foundation Model: Foundation AI Models for VLSI Circuit Design and EDA

arXiv:2504.03711v2 Announce Type: replace-cross Abstract: Artificial intelligence (AI)-driven electronic design automation (EDA) techniques have been extensively explored for VLSI circuit design appli

tutorialsarxiv-cs-lg

A^{2}utoLPBench: An Auto-Generated, Agent-Friendly LP Benchmark via Inverse-KKT Construction

arXiv:2607.02141v1 Announce Type: new Abstract: Most LP-from-text benchmarks are static datasets of word problems written and labeled by hand. Once such a dataset is released, its size is fixed, its d

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

arXiv:2604.08169v2 Announce Type: replace Abstract: Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent misalignme

Adaptive Companionship for Group-Following Robots: Handling Dynamically Changing Group Formations

arXiv:2607.01287v1 Announce Type: cross Abstract: Accompanying a group of humans is an essential aspect of developing human-like social cognition in robots. However, human groups typically do not foll

AI Assistance for Human Review of Default Judgments

arXiv:2607.01256v1 Announce Type: cross Abstract: Overwhelmed courts in the United States review millions of default judgments each year. Unfortunately, such manual reviews are time-consuming and pron

applicationsarxiv-cs-ai

AI Virtue: What is 'Good' Knowledge in the Age of Artificial Intelligence?

arXiv:2607.01776v1 Announce Type: cross Abstract: In the age of AI, what will be good knowledge? This article, which is accepted and forthcoming in a special issue of Modern Fiction Studies on 'Cultur

Agents

Aria: An Agent For Retrieval and Iterative Auto-Formalization via Dependency Graph

arXiv:2510.04520v2 Announce Type: replace Abstract: Accurate auto-formalization of theorem statements is essential for advancing automated discovery and verification of research-level mathematics, yet

agentsarxiv-cs-ai

ART for Diffusion Sampling: Continuous-Time Control and Actor-Critic Learning

arXiv:2607.02137v1 Announce Type: cross Abstract: We study timestep allocation for score-based diffusion sampling, where a learned reverse-time dynamics is discretized on a finite grid. Uniform and ha

Artificial Intelligence-Enabled Accounting Information Systems and Fraud Detection in Nigeria's Financial Services Sector: The Moderating Role of Natural Language Processing

arXiv:2607.01257v1 Announce Type: cross Abstract: The rapid digitalisation of financial systems has improved operational efficiency and financial inclusion while simultaneously increasing exposure to

Autorelevance function and other feature relevance measures for univariate time series

arXiv:2607.01959v1 Announce Type: cross Abstract: We propose a model agnostic methodology to measure lag relevance in machine learning forecasting models applied to univariate time series. Particularl

researcharxiv-cs-lg

Black-Box Inference of LLM Architectural Properties with Restrictive API Access

arXiv:2607.01313v1 Announce Type: cross Abstract: In practice, most commercial LLM providers do not publicly release details of underlying LLM architectures. However, prior work has shown that given l

Local Ai

Bridge-WA: Predicting Where and How the World Changes for Robotic Action

arXiv:2607.02195v1 Announce Type: new Abstract: General-purpose vision-language-action models benefit from large vision-language priors, but effective manipulation also requires anticipating action-re

local-aiarxiv-cs-ro

CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

arXiv:2603.22435v2 Announce Type: replace-cross Abstract: 'Code-as-Policy' considers how executable code can complement data-intensive Vision-Language-Action (VLA) methods, yet their effectiveness as

CNN Models for Microphone Array Covariance Matrix Upsampling and Acoustic Imaging

arXiv:2607.01295v1 Announce Type: cross Abstract: Acoustic imaging visualization is a core methodology in acoustics, enabling spatial analysis of sound sources and acoustic scenes. However, limited se

applicationsarxiv-cs-lg

Agents

Coding-agents can replicate scientific machine learning papers

arXiv:2607.02134v1 Announce Type: new Abstract: Scientific machine learning papers typically make computational claims, e.g., that the relative mean square error is less than 5% or that the 95% predic

agentsarxiv-cs-ai

CoFL-S: Spatially Queryable Sector Flow Fields for Local Language-Conditioned Navigation

arXiv:2607.02222v1 Announce Type: cross Abstract: Vision-Language Navigation has increasingly emphasized high-level instruction reasoning, memory, global map construction, and instruction decompositio

COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows

arXiv:2607.01709v1 Announce Type: new Abstract: Agents are increasingly used to construct workflows and assist humans in completing recurring tasks more efficiently. As these workflows become repeated

Composite Reward Design in PPO-Driven Adaptive Filtering

model-releasesarxiv-cs-lg

arXiv:2506.06323v2 Announce Type: replace-cross Abstract: Model-free and reinforcement learning-based adaptive filtering methods are gaining traction for denoising in dynamic, non-stationary environme

Conditional Inference Trees and Forests for Feature Selection

arXiv:2607.01417v1 Announce Type: new Abstract: Conditional inference trees (CIT) and conditional inference forests (CIF) reduce split-selection bias by testing features before choosing split threshol

safetyarxiv-cs-lg

ContextSniper: AntTrail's Token-Efficient Code Memory for Repository-Level Program Repair

arXiv:2607.01916v1 Announce Type: new Abstract: Large language model agents can repair real repository issues, but they often spend large context budgets on whole-file reads, broad searches, and long

Copewell: A Multi-Agent Swarm Architecture for Equitable Mental Wellness Support

arXiv:2607.02245v1 Announce Type: new Abstract: Mental health disorders affect nearly one billion people globally, yet 75% of individuals in low- and middle-income countries receive no treatment due t

Hardware

DeadPool: Resilient LLM Training with Hot-Swapping via Zero-Overhead Checkpoint

arXiv:2607.01646v1 Announce Type: new Abstract: State-of-the-art large language model (LLM) training takes tens of thousands of graphics processing units (GPUs) for months and encounters failures acro

hardwarearxiv-cs-lg

Decomposer: Learning to Decompile Symbolic Music to Programs

arXiv:2607.01849v1 Announce Type: cross Abstract: Musical performance involves executing a set of high-level musical instructions, yet recovering those instructions from the performance is a challengi

applicationsarxiv-cs-ai

DecompRL: Solving Harder Problems by Learning Modular Code Generation

model-releasesarxiv-cs-lg

arXiv:2607.02390v1 Announce Type: new Abstract: How can Large Language Models (LLMs) solve problems they currently cannot? Repeated sampling scales test-time compute but GPU cost grows linearly with a

DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agents

arXiv:2607.01557v1 Announce Type: cross Abstract: Large Language Models (LLMs) often struggle with persuasion in high-stakes scenarios. People's individual personalities and concerns require tailored

DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models

arXiv:2607.02374v1 Announce Type: new Abstract: Personalization changes what a model says to a user; we show that it can also change the reasoning trajectory used to justify the response. Modern LLMs

Embedding Inference Attack

arXiv:2607.01276v1 Announce Type: cross Abstract: Embedding models are essential components of modern Information Retrieval (IR) systems, yet they are typically hidden behind APIs. Recent works have s

researcharxiv-cs-lg