Search: “evaluation”

Does Machine Translation Retain Media Bias?

arXiv:2610.19945v1 Announce Type: new Abstract: Within the most recent two decades, machine translation (MT) has been advanced from a research area to a mature, deployed technology for prose text and

safetyarxiv-cs-cl

Does the Linguistic Register of Thinking Matter? Wenyan as a Compact Reasoning Register for Chinese LLMs

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2610.19741v1 Announce Type: new Abstract: Visible reasoning traces of Chinese reasoning-oriented LLMs are typically written in modern vernacular Chinese, which may be verbose under limited decod

model-releasesarxiv-cs-cl

15 Oct 2026

EmoXpt: Analyzing Emotional Variances in Human Comments and LLM-Generated Responses

arXiv:2501.06597v1 Announce Type: cross Abstract: The widespread adoption of generative AI has generated diverse opinions, with individuals expressing both support and criticism of its applications. T

researcharxiv-cs-ai

Empirical Boundaries of Reservoir Computing for Few-Shot Sequence Pattern Tasks

arXiv:2610.19997v1 Announce Type: new Abstract: Efficient sequence modeling remains a fundamental challenge in machine learning. Despite the success of Transformers, recurrent neural networks (RNNs) h

EvoCowork: A Benchmark for Real-World Office Tasks

arXiv:2610.14828v1 Announce Type: new Abstract: Office workflows are a demanding testbed for self-improving agents because their deliverables must be correct, natively editable, and mutually consisten

FlowSep-RF: Rectified Flow Matching for Single-Channel Radio-Frequency Signal Separation

arXiv:2610.20269v1 Announce Type: cross Abstract: Single-channel radio-frequency (RF) signal separation is important for interference mitigation when a signal of interest overlaps with a co-channel in

From Efficiency to Sufficiency: The Cost-Aware Measurement Design Framework for Automated Essay Scoring

arXiv:2610.19287v1 Announce Type: cross Abstract: Compute cost in automated essay scoring (AES) is usually treated as a technical efficiency issue rather than a measurement design concern. Yet deploym

researcharxiv-cs-cl

From Unreliable to Usable: Decomposing Failure Modes and Confidence Signals in LLM Entity Extraction - Evidence from Cyber Threat Intelligence

model-releasesarxiv-cs-cl

arXiv:2610.20417v1 Announce Type: new Abstract: Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures ac

Game-Creation-Agents: Autonomous Game Production

arXiv:2610.16776v1 Announce Type: new Abstract: Game-Creation-Agents (GCA) turns a natural-language request into a runnable Phaser 4 browser game. The pipeline first builds a multimodal game design do

GDEPO: Group Dual-dynamic and Equal-right Advantage Policy Optimization for Sample-Efficient Reinforcement Learning

arXiv:2601.06795v4 Announce Type: replace Abstract: Automated Theorem Proving (ATP) is a fundamental AI challenge, requiring machine-verifiable proofs in formal languages like Lean to evaluate reasoni

safetyarxiv-cs-ai

GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to spe

GPT-Rosalind is built to support that work: helping researchers reason across scientific literature and biological data, with scientists bri…

GPT-Rosalind is built to support that work: helping researchers reason across scientific literature and biological data, with scientists bringing the expertise and judgment to evaluate the evidence. E

model-releasesopenai--x

HGRP: Heterogeneous Graph Neural Network for Routability Prediction

arXiv:2610.20209v1 Announce Type: new Abstract: Routability prediction enables rapid evaluation of placement quality without routing, providing effective feedback for routability-driven placement opti

Historical-Feature Residual Adaptation for Sequential Screening

arXiv:2610.19315v1 Announce Type: new Abstract: Sequential screening policies must use acquired feedback effectively under a fixed experiment budget. We study acquisition-time residual adaptation: a s

safetyarxiv-cs-lg

Applications

Horizon Scans can be accelerated using novel information retrieval and artificial intelligence tools

arXiv:2504.01627v2 Announce Type: replace-cross Abstract: Introduction: Horizon scanning in healthcare assesses early signals of innovation, crucial for timely adoption. Current horizon scanning faces

applicationsarxiv-cs-ai

KoAGE: Do Large Language Models Know, Learn, and Coin Korean Neologisms?

model-releasesarxiv-cs-cl

arXiv:2610.19399v1 Announce Type: new Abstract: Korean neologisms arise through diverse mechanisms, a demanding test of language models beyond lexical memorization. To enable systematic evaluation, we

Last Secure Code Benchmark: Is the Code You Vibe Secure?

arXiv:2610.14766v1 Announce Type: cross Abstract: AI coding agents now write much of the code that ships, yet whether that code is secure remains unclear. We introduce the Last Secure Code Benchmark,

LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity

arXiv:2610.18092v1 Announce Type: new Abstract: Deep research (DR) agents address open-ended information needs through multi-step reasoning, long-horizon web search, and evidence synthesis. However, e

LLM-Based Policy Compliance Verification of Loan Applications

arXiv:2610.20155v1 Announce Type: cross Abstract: Banks rely on extensive internal policy frameworks to ensure that corporate lending decisions comply with regulatory requirements and internal risk st

safetyarxiv-cs-cl

LLM-jp-4 Technical Report

arXiv:2610.19331v1 Announce Type: new Abstract: This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, a

researcharxiv-cs-cl

LLMRec: Benchmarking Large Language Models on Recommendation Task

arXiv:2308.12241v2 Announce Type: replace-cross Abstract: Recently, the fast development of Large Language Models (LLMs) such as ChatGPT has significantly advanced NLP tasks by enhancing the capabilit

MagicGarment: Benchmarking Single-Image Standalone 3D Garment Reconstruction

model-releasesarxiv-cs-cv

arXiv:2610.19623v1 Announce Type: new Abstract: Single-image 3D reconstruction has advanced rapidly, yet standalone garments remain challenging due to thin-shell geometry, open boundaries, and layered

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

arXiv:2610.19817v1 Announce Type: new Abstract: Human values shape how we expect language models to behave. Dominant alignment approaches treat values as global objectives, although the appropriate va

safetyarxiv-cs-cl

Neural POS Tagging for the Low-Resource Saraiki

model-releasesarxiv-cs-cl

arXiv:2610.20279v1 Announce Type: new Abstract: Part-of-speech tagging is a fundamental sequence labeling task and an important component for many downstream natural language processing applications.

nvidia/Nemotron-3.5-Embed-1B-BF16 · Hugging Face

model-releasesr-localllama

Nemotron-3.5-Embed-1B-BF16 is a versatile embedding model developed by NVIDIA for multimodal and multilingual question-answering retrieval. The model can embed a document page in the form of an image,

Paraphrase Consensus Is Not Self-Consistency: Instruction Reliance and Robustness in VLA Policies

arXiv:2610.19438v1 Announce Type: new Abstract: Vision-language-action (VLA) policies fine-tuned on LIBERO are trained and evaluated on the same ten instruction strings, and rewording the instruction

safetyarxiv-cs-ro

Passing Tests, Missing Semantics: Evidence of Interpretive Lock-In in Plan-Based DataFrame Code Generation

arXiv:2610.15797v1 Announce Type: new Abstract: Planning and revision are intended to bring generated code closer to the task specification. Yet a mistaken interpretation can persist across the plan,

PatientWM: Towards Generalizable Clinical World Models via Unified Patient States

arXiv:2610.17059v1 Announce Type: new Abstract: Clinical decisions depend on anticipating how a patient's condition will evolve as treatment is adjusted and new evidence becomes available. Existing me

researcharxiv-cs-ai

Quantifying privacy risks in foundation model representations of brain MRI

model-releasesarxiv-cs-cv

arXiv:2610.20494v1 Announce Type: new Abstract: Pretrained representations are increasingly used to share and reuse information from medical images without exposing the original scans, but it is uncle

RASSG: Repair-Aware Semantic Survival Gating for Streaming Tool Execution

arXiv:2610.14659v1 Announce Type: new Abstract: Streaming speech assistants can launch tools before users finish speaking, overlapping execution with incoming speech to reduce user waiting time. Howev

Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation

model-releasesarxiv-cs-cv

arXiv:2610.20335v1 Announce Type: new Abstract: Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction, a

Recursive Environment Scaling for General Long-Horizon Agents

arXiv:2610.16397v1 Announce Type: new Abstract: We study environment scaling as a way to increase agent-benchmark difficulty while holding core task objectives fixed. Our framework, Recursive Environm

Local Ai

Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness

arXiv:2610.20543v1 Announce Type: new Abstract: Multimodal sentiment models are usually evaluated on complete utterances, yet streams often lose a contiguous span of text, audio, or video. We study lo

local-aiarxiv-cs-cv

Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows

arXiv:2610.17305v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently improved Text-to-SQL generation, but their computational cost can hinder deployment in resource-constrained s

Risk-Controlled DFT Selection for Machine-Learned Interatomic Potentials

arXiv:2610.19630v1 Announce Type: cross Abstract: Automated materials discovery requires evaluating large pools of candidate configurations, with density functional theory (DFT) as a major throughput

Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers

arXiv:2610.17820v1 Announce Type: cross Abstract: TypeSafe AI's Jev is a new kind of safety judge. It generates no text and no label: it answers questions the developer defines in advance and returns

ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments

arXiv:2606.20235v2 Announce Type: replace-cross Abstract: Academic paper search is a core step in scientific research, and LLM-based search agents are emerging as a promising paradigm for iterative, i

SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generation with LLMs

arXiv:2508.08742v3 Announce Type: replace-cross Abstract: Scientific literature question answering is a pivotal step towards new scientific discoveries. Recently, extit{two-stage} retrieval-augmented

SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition

arXiv:2610.14966v1 Announce Type: new Abstract: Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluat

Local Ai

SnapFlow: One-Step Action Generation for Flow-Matching VLAs via Progressive Self-Distillation

arXiv:2604.05656v2 Announce Type: replace Abstract: Vision-language-action (VLA) models with flow-matching action heads, such as {pi}0 and {pi}0.5, have become a standard recipe for generalist robot m

local-aiarxiv-cs-cv

Tutorials

Spectral structure learning for clinical time series

arXiv:2502.11680v2 Announce Type: replace Abstract: We develop and evaluate a structure learning algorithm for clinical time series. Clinical time series are multivariate time series observed in multi

tutorialsarxiv-cs-lg

Structure-Preserving Document-Level Data Augmentation via Discourse Parsing

arXiv:2610.19497v1 Announce Type: new Abstract: Data augmentation (DA) plays a vital role in improving model performance in low-resource or limited-supervision scenarios within Natural Language Proces

researcharxiv-cs-cl

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery Blog: https://pub.sakana.ai/agentic-scientist/ Gi…

model-releasesdavid-ha--x

The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery Blog: https://pub.sakana.ai/agentic-scientist/ GitHub: https://github.com/SakanaAI/AI-Scientist-v3 arXiv: htt

ThreadBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions

arXiv:2606.14715v2 Announce Type: replace-cross Abstract: LLM agents are increasingly used to simulate real-world interactions, but it remains unclear whether simulated behaviors preserve the content

Unconstrained CTI-REALM with Open-Weights

arXiv:2610.14180v1 Announce Type: new Abstract: Organisations that want language-model agents for security work face a choice between open-weight models, which they can run and audit themselves, and f

Local Ai

What We Talk About When We Talk About LLM Planning: Evidence for Two Distinct Planning Abilities

arXiv:2607.11197v2 Announce Type: replace Abstract: When LLMs exhibit uneven performance across planning tasks, these gaps are often attributed to task difficulty. We argue that this explanation is in

local-aiarxiv-cs-ai

When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents

arXiv:2606.22936v2 Announce Type: replace Abstract: Tool-using LLM agents make early choices about what to search for, which evidence to trust, and what to do next. A wrong early choice can steer ever

When Pleasing the User Compromises Care: AI Sycophancy in Caregiver-Mediated Healthcare

arXiv:2610.14360v1 Announce Type: new Abstract: Large language model (LLM) alignment often assumes that the user interacting with a model is also the primary beneficiary of its behavior. Yet in many c