Search: “evaluation”

Quantum Kernel Estimation for the Discovery of Early Lung Cancer Detection

arXiv:2608.19304v1 Announce Type: cross Abstract: Lung cancer screening with low-dose chest computed tomography reduces mortality, but its impact is limited by uptake, adherence, and management challe

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Repo0: Design-Driven Zero-to-All Code Generation

DGX agent

arXiv:2608.19854v1 Announce Type: cross Abstract: Large language model agents have made substantial progress in code generation, yet most existing systems assume a predefined repository architecture.

model-releasesarxiv-cs-ai

21 Aug 2026

SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?

arXiv:2608.19799v1 Announce Type: new Abstract: Software increasingly functions as part of the scientific instrument itself, making failures in scientific code capable of compromising not only program

TempJail: Temporal Jailbreak Attack against Large Vision-Language Models via Subtitle Scheduling

arXiv:2608.19737v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) have achieved remarkable progress in video understanding and reasoning. Despite extensive studies on text- and im

Towards Quantifying Benchmark Optimization in ASR Models

arXiv:2608.19936v1 Announce Type: cross Abstract: Public benchmarks are important measures of Automatic Speech Recognition (ASR) model capabilities. However, by nature of being public, there is risk o

Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning

arXiv:2608.20284v1 Announce Type: new Abstract: Reliable surgical planning requires models to anticipate not only how instruments will move, but also how the operative visual state will evolve togethe

researcharxiv-cs-cv

When Guidance Goes Off-Scale: Recalibrating Diffusion Transformers under Analog Compute-in-Memory Nonidealities

arXiv:2608.19644v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) incur high memory traffic and energy costs because sampling repeatedly evaluates large denoisers dominated by linear opera

researcharxiv-cs-cv

When Saying No Makes Better Videos: Designing Dual Gatekeeping for Pedagogically Grounded AI Content Creation

arXiv:2608.19812v1 Announce Type: new Abstract: To prevent the adoption of aesthetically polished but pedagogically flawed AI content, we study a video authoring pipeline featuring two layers of struc

Which Eviction Policy Should an LLM Cache Use? A Systematic Study Across Workloads, Capacities, and Encoders

arXiv:2608.20280v1 Announce Type: cross Abstract: Semantic caches reuse an LLM response when the incoming query embedding lies near a cached query, but proposed eviction policies have rarely been comp

safetyarxiv-cs-lg

Agents

A Multi-Agent Platform for Automated Enterprise Analytics and Insight Generation

arXiv:2608.18740v1 Announce Type: new Abstract: This paper proposes a multi-agent framework built on CrewAI [1] for conversational business intelligence. Five specialized AI agents operate in a sequen

agentsarxiv-cs-ai

A systematic review of machine learning techniques to address diagnosis and treatment of autism: challenges and opportunities

arXiv:2608.18188v1 Announce Type: cross Abstract: Autism spectrum disorder (ASD) is a developmental disability characterized by challenges in social interaction and communication. As the causes of ASD

Aslema at NADI 2026: Augmentation through Fewshot for SLU

arXiv:2608.18689v1 Announce Type: cross Abstract: We present Aslema, our system for NADI 2026 Shared Task 5, which consists of two subtasks: intent recognition and slot filling. We evaluate four omni

DeepTCM1.0: A Multi-Expert AI Agent for Deciphering Mechanisms of Chinese Herbal Formulae Based on General Large Language Models

arXiv:2608.18103v1 Announce Type: cross Abstract: Background: Mechanistic elucidation of traditional Chinese medicine (TCM) compound formulas remains a central challenge in the modernization of TCM. C

FairGlucose: A CGM Fairness Benchmark Reveals Subgroup Disparities Hidden in Population-Level Validation

arXiv:2608.18296v1 Announce Type: cross Abstract: As CGM-based AI tools approach clinical deployment, whether their accuracy is equitable across patient demographics remains insufficiently tested. To

FM-Bench: A Benchmark for Long-Horizon Management with Competing Agents

arXiv:2608.18423v1 Announce Type: new Abstract: Language model agents now execute bounded tasks reliably. Whether they can sustain effective decision-making over long horizons, where actions have cumu

How Quantum Is the Advantage? A Fair, Calibration- and Noise-Aware Benchmark and Attribution Audit of Quantum Machine Learning for Network Intrusion Detection

arXiv:2608.18155v1 Announce Type: cross Abstract: Quantum machine learning (QML) for network intrusion detection (NIDS) is routinely reported to reach near-perfect accuracy, yet the most rigorous stud

MedUAG: Unified Understanding and Generation for Medical Multimodal Models

arXiv:2608.18937v1 Announce Type: cross Abstract: Recent Multimodal Large Language Models (MLLMs) are rapidly evolving into unified understanding and generation (UAG) frameworks. However, extending th

Applications

Optimizing Energy Efficiency and Grid Stability via Public EV Charging Flexibility

arXiv:2608.18126v1 Announce Type: cross Abstract: This study evaluates the potential of electric vehicle (EV) charging flexibility to enhance both energy efficiency and power grid stability. Using rea

applicationsarxiv-cs-lg

Persona-Guided LLM Agents for Task-Oriented Dialogue

arXiv:2608.18085v1 Announce Type: new Abstract: Prior work has shown that large language models (LLMs) can express diverse personality traits in open-ended text generation. However, it remains unclear

Position: Current Model Cards Are Insufficient for Downstream Governance of Open-Weight Foundation Models

arXiv:2608.18086v1 Announce Type: new Abstract: The growth of open-weight foundation models (OWFMs) has prompted the AI community to re-evaluate strategies for effective downstream governance. Althoug

Agents

Self-Improvement of Large Language Models: A Technical Overview and Future Outlook

arXiv:2603.25681v2 Announce Type: replace Abstract: As large language models (LLMs) continue to advance, improving them solely through human supervision is becoming increasingly costly and limited in

agentsarxiv-cs-cl

Solving Is Not Drawing: A Benchmark for Diagrammatic Reasoning in Olympiad Geometry

arXiv:2608.18111v1 Announce Type: new Abstract: Foundation models such as GPT and Claude now solve olympiad-level mathematics with remarkable proficiency, so much so that geometry problem solving has

Structure, Association, and Decision Value: Representation-Based Difficulty Estimation for Adaptive Inference in African-Language NLI

arXiv:2608.19003v1 Announce Type: new Abstract: We ask whether internal representation statistics can provide useful example-level difficulty signals for adaptive inference in multilingual African NLP

Tangut Word Segmentation under Extreme Resource Scarcity: Integrating Traditional Lexicons and Unlabeled Text

arXiv:2608.18437v1 Announce Type: new Abstract: Tangut is an extinct language whose script does not explicitly mark word boundaries. We present the first systematic study of Tangut word segmentation u

researcharxiv-cs-cl

The Impact of CutMix on Reliability and Robustness in Semantic Segmentation

arXiv:2608.18715v1 Announce Type: cross Abstract: Ensuring not only high accuracy but also reliable and robust predictions is critical for the deployment of semantic segmentation models in safety-crit

VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

model-releasesarxiv-cs-cv

arXiv:2608.18607v1 Announce Type: new Abstract: Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combi

VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on

model-releasesarxiv-cs-cv

arXiv:2601.02945v2 Announce Type: replace Abstract: With the rapid development of e-commerce and digital fashion, image-based virtual try-on (VTON) has attracted increasing attention. However, existin

What is Missing from AI Post-Training AI: An Empirical Analysis

arXiv:2608.19072v1 Announce Type: new Abstract: Large language model (LLM) agents can now post-train an LLM end-to-end. They can write code, launch training, evaluate checkpoints, and improve downstre

A decodability criterion predicts when hidden-state selection beats majority voting in large language models

arXiv:2608.17124v1 Announce Type: new Abstract: Combining the answers a large language model (LLM) samples for a question into one decision is a test-time information fusion problem, usually solved by

Advancing Health Equity through Multi-Level Fairness in Health Informatics

arXiv:2608.16902v1 Announce Type: cross Abstract: The increasing integration of machine learning in healthcare has highlighted critical challenges related to fairness, transparency, and health equity.

safetyarxiv-cs-lg

An Empirical Study of Reward Specification and Benchmark Reliability in GRPO-based LLM Unlearning

arXiv:2608.17804v1 Announce Type: cross Abstract: Practical LLM unlearning is usually evaluated through two objectives: suppress target-specific knowledge and preserve non-target utility. In generativ

ASI-Bench: At the Dawn of Artificial Superintelligence

arXiv:2608.17271v1 Announce Type: new Abstract: Artificial superintelligence (ASI) requires AI to move beyond mastering existing knowledge toward exploring the unknown, creating new knowledge, and tur

BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models

arXiv:2608.17895v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) have made significant strides in visual comprehension, their ability to reason about text-dense, profes

Can Large Language Models Explain Flight Safety Events? A Prior-Guided Semantic LLM-based Approach

arXiv:2608.18017v1 Announce Type: new Abstract: Improving flight safety with flight data requires not only accurate detection of risk events, but more importantly, clear interpretation of their underl

Digital Twin-Based Intrusion Detection for Vehicle Powertrain CAN Bus Systems

arXiv:2608.17093v1 Announce Type: cross Abstract: Existing automotive intrusion detection systems (IDSs) for the Controller Area Network (CAN) largely target discrepancies in message timing, frequency

researcharxiv-cs-lg

Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models

arXiv:2604.00547v2 Announce Type: replace Abstract: Unified Multimodal Large Models (UMLMs) integrate understanding and generation capabilities within a single architecture. While unified architecture

Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts

researchapple-ml-research

Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining

Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing

arXiv:2509.12040v3 Announce Type: replace-cross Abstract: Open-Vocabulary Remote Sensing Image Segmentation (OVRSIS), an emerging task that adapts Open-Vocabulary Segmentation (OVS) to the remote sens

Tutorials

From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation

arXiv:2608.18076v1 Announce Type: cross Abstract: Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically o

tutorialsarxiv-cs-ai

Grading Needs a Rubric, Not Intelligence

arXiv:2608.17938v1 Announce Type: cross Abstract: Small language models can grade open-ended examination answers as reliably as substantially more expensive models when they grade against an explicit

MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps

arXiv:2608.17659v1 Announce Type: cross Abstract: LLM-powered GUI agents that autonomously operate smartphones are rapidly transitioning from research prototypes to early real-world deployment. Howeve

PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contexts

model-releasesarxiv-cs-cv

arXiv:2608.17607v1 Announce Type: new Abstract: Whole-slide pathology reasoning requires models to integrate gigapixel-scale visual evidence across complete case-linked slides, yet current question-an

Revisiting WEASEL 2.0: Reproduction, Sensitivity, and an Adaptive Ensemble-Size Rule

arXiv:2608.18021v1 Announce Type: new Abstract: WEASEL 2.0 is a dictionary-based time series classifier that combines dilated sliding windows with a randomised hyperparameter ensemble and a fixed-size

researcharxiv-cs-lg

Risk-Controllable Multi-View Diffusion for Driving Scenario Generation

arXiv:2603.11534v2 Announce Type: replace Abstract: Generating safety-critical driving scenarios is crucial for evaluating and improving autonomous driving systems, but long-tail risky situations are

safetyarxiv-cs-cv

Runtime Governance for Agentic AI: Action-Boundary Control with Trusted Provenance and Fail-Closed Execution

arXiv:2608.16891v1 Announce Type: new Abstract: Agentic AI systems request tool actions that can modify files, send messages, launch jobs, or change workflow state. This shifts the safety problem from

Same GRPO recipe on three from-scratch LLMs (353M/316M/672M) gave three different outcomes, with no clean relationship to scale [P]

model-releasesr-machinelearning

I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparam

Agents

SkillEffect: Checked Lowering for Memory-Bounded Agent Tools

arXiv:2608.17007v1 Announce Type: new Abstract: Agent Skills can specify procedural and resource obligations for tool use, and language models instantiate them as concrete programs. However, when mode

agentsarxiv-cs-ai

SPSA Hyperparameter Tuning for Variational Quantum Natural Language Inference

model-releasesarxiv-cs-lg

arXiv:2608.16939v1 Announce Type: cross Abstract: Training variational quantum models requires choosing between parameter-shift gradients, which are exact but cost O(P) forward evaluations, and simult