Search: “open-weights”

Who Judges Matters: Measuring Family-Conditioned Preference in LLM-as-Judge Panels

arXiv:2609.17857v1 Announce Type: cross Abstract: Who the judge is can affect an LLM-as-judge result, but measuring that effect without confusing it with candidate quality is difficult. We study four

17 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Beyond the Name: Demographic Leakage in De-Identified Resumes and Evaluation Artifacts in LLM Bias Audits

DGX agent

arXiv:2609.16501v1 Announce Type: cross Abstract: De-identified resume screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leak

safetyarxiv-cs-ai

16 Sept 2026

BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents

arXiv:2609.16305v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

arXiv:2609.16800v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remai

safetyarxiv-cs-cl

The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It

arXiv:2609.16247v1 Announce Type: new Abstract: Large language models sometimes behave in ways resembling human emotional responses, and recent work has identified internal representations that may ex

When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

arXiv:2609.17516v1 Announce Type: cross Abstract: Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-

Calibrating Interpretability Instruments Before Trusting Their Verdicts

arXiv:2609.14754v1 Announce Type: cross Abstract: Causal claims about large language model (LLM) internals rest on measurements. Those might include a projection, a cosine, an ablation delta, or an in

Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning

arXiv:2511.13368v3 Announce Type: replace-cross Abstract: Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and

Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models

arXiv:2609.13816v1 Announce Type: cross Abstract: JavaScript powers approximately 98.8% of all websites, making vulnerabilities in its code a significant security risk, yet existing detection approach

Grounded Adjudication of Variations across Extracted TimeLines (GAVEL): Comparing Clinical Timelines Against Their Case Reports

arXiv:2609.13475v1 Announce Type: new Abstract: Existing pipelines for clinical timeline extraction from case reports are evaluated using an expert reference and are limited by imperfect reference ann

Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity

arXiv:2603.29396v2 Announce Type: replace Abstract: Standard evaluations of Large language models (LLMs) focus on task performance, offering limited insight into whether correct behavior reflects appr

Mizan: A National Benchmark for Evaluating Large Language Models on Iraqi Arabic and the Iraqi Civic Context

arXiv:2609.13980v1 Announce Type: cross Abstract: Arabic large-language-model (LLM) evaluation has matured around Modern Standard Arabic (MSA): aggregated leaderboards such as the Open Arabic LLM Lead

Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Families

arXiv:2609.14759v1 Announce Type: cross Abstract: Alignment applied after pretraining is shallow in a measurable way: a single direction in a model's residual stream can be edited out, and the model s

Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs

arXiv:2609.13582v1 Announce Type: cross Abstract: A clinical agent benchmark can report the same verdict on identical inputs while the agent files a materially different order on each run. Such agents

Temperature Fragility and the Conditional Benefits of Truncation Sampling

arXiv:2609.15476v1 Announce Type: new Abstract: Large language models generate text by sampling each token from a predicted distribution, and a temperature parameter sets how far the draw strays from

The Misery of Mechanistic Interpretability: A Formal Perspective

arXiv:2609.15533v1 Announce Type: cross Abstract: Mechanistic interpretability has become the dominant lens for understanding frontier language models, as their inner workings are complex and inherent

Trustworthy, Explainable, and Sustainable Decentralized Intelligence for 6G Networks

arXiv:2609.13872v1 Announce Type: cross Abstract: As 6G networks transition from theoretical frameworks into operational realities, artificial intelligence (AI) evolves from an add-on optimization too

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

arXiv:2609.15028v1 Announce Type: new Abstract: In-context learning (ICL) enables models to infer tasks from demonstrations, but existing benchmarks generally lack matched text and image versions need

When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings

arXiv:2609.13824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to evaluate the responses of other language models. This approach, known as LLM-as-a-Judge, is faster

BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines

arXiv:2604.05793v2 Announce Type: replace-cross Abstract: In LLM agent pipelines, prompt privacy risk propagates beyond a single model call: raw user content enters retrieval queries, memory writes, t

ChitraMiti: Benchmarking Visual Grounding and Modality Reliance in Bengali Geometric Reasoning

arXiv:2609.12509v1 Announce Type: new Abstract: Evaluation of vision-language models (VLMs) for multimodal mathematical reasoning remains limited for low-resource languages and for geometry problems t

Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

arXiv:2609.12243v1 Announce Type: new Abstract: Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-tra

researcharxiv-cs-cl

Measuring Pragmatic Influence in Large Language Model Instructions

arXiv:2602.21223v2 Announce Type: replace Abstract: It is not only what we ask large language models (LLMs) to do that matters, but also how we ask them. Phrases like ``This is urgent'' or ``As your s

researcharxiv-cs-cl

MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

arXiv:2609.12884v1 Announce Type: new Abstract: A medical claim's correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range,

Can LLMs Follow Medical Expert Logic? A Benchmark for Hierarchical Logical Consistency in Risk-of-Bias Assessment

arXiv:2609.11185v1 Announce Type: new Abstract: Evidence-based medicine demands strict logical consistency, yet current evaluations of large language models (LLMs) prioritize superficial label matchin

Agents

Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification

arXiv:2609.11319v1 Announce Type: new Abstract: Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLM

agentsarxiv-cs-ai

Refusal Beyond a Single Direction: A Preliminary Comparison of Diff-in-Means and INLP

arXiv:2606.13720v2 Announce Type: replace Abstract: Arditi et al. (2024) has shown that refusal in safety fine-tuned chat models is mediated by a single linear direction in the residual stream, recove

safetyarxiv-cs-ai

Rescaling Confidence: What Scale Design Reveals About LLM Metacognition

arXiv:2603.09309v3 Announce Type: replace Abstract: Verbalized confidence, in which LLMs report a numerical certainty score, is widely used to estimate uncertainty in black-box settings, yet the confi

Towards a Deterministic Math Solver for Clinical Language Models

arXiv:2609.10728v1 Announce Type: new Abstract: Large language models are unreliable at arithmetic, which is a problem for clinical calculators where a single numerical error changes the recommendatio

BodyCam-VQA: Enhanced Body-Worn Camera Video Captioning via Multimodal Reasoning and Probe Question Generation

arXiv:2609.10815v1 Announce Type: cross Abstract: Police body-worn camera (BWC) footage has emerged as a critical aspect of law enforcement that ensures legal transparency, officer accountability, and

safetyarxiv-cs-cl

Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs

arXiv:2609.10413v1 Announce Type: new Abstract: Current LLM memory systems treat all personal facts identically, so stores grow without bound while retrieval precision degrades. The core challenge is

OpenDiscoveryTrace: Process Traces for Evaluating AI Scientist Workflows

arXiv:2609.09203v1 Announce Type: new Abstract: Existing benchmarks for autonomous AI scientists evaluate only final outputs---generated code, hypotheses, or papers---yet discard the reasoning process

Watermarks Without Verification: AI Text Watermarking After the EU AI Act

arXiv:2609.09604v1 Announce Type: cross Abstract: On August 2, 2026, the obligations of Article 50 of the EU AI Act took effect, requiring generative AI providers to mark the content their systems pro

Local Ai

(Whose defaults?) Is artificial intelligence reorienting archaeological methods?

arXiv:2609.11198v1 Announce Type: cross Abstract: Generative AI and the practice of 'vibe coding' are changing how archaeologists carry out computational research, but their effects on the discipline'

local-aiarxiv-cs-cl

A Layered Analysis of Disagreement And Answer Quality in Multi-Agent LLM Debate

arXiv:2609.08016v1 Announce Type: new Abstract: Multi-agent debate, in which several LLMs exchange arguments before answering, is widely assumed to improve answer quality by surfacing genuine disagree

safetyarxiv-cs-ai

A Translational Note on AI Safety Evaluation

arXiv:2609.06573v1 Announce Type: new Abstract: Recent studies report that automated red-teaming finds more vulnerabilities, at lower cost, than human red-teaming on standard AI safety benchmarks, and

Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks

arXiv:2609.08765v1 Announce Type: cross Abstract: Large language model (LLM) benchmarks are often treated as fixed datasets with stable scores, yet their outcomes depend on configurable evaluation pip

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

arXiv:2609.09410v1 Announce Type: new Abstract: While autonomous agents have made significant strides in 'deep research' by iteratively navigating the open web to synthesize information, real-world pr

Evidence-Grounded Retrieval for Investigation Hunt Lead Generation from CTI Reports

arXiv:2609.08790v1 Announce Type: cross Abstract: Threat hunting increasingly depends on converting unstructured knowledge (e.g., Cyber Threat Intelligence reports) into actionable hunt leads: concise

FreqBLiMP: Frequency-Controlled Minimal Pairs Reveal Robustness and Fragility of LLMs Under Lexical Rarity

arXiv:2609.07153v1 Announce Type: cross Abstract: Minimal-pair benchmarks such as BLiMP evaluate linguistic knowledge by testing whether language models (LMs) prefer acceptable sentences over minimall

How Fragile Is Safety Alignment at Frontier Scale? A Single-Direction Attack on a 320B MoE

arXiv:2609.09793v1 Announce Type: cross Abstract: Directional ablation removes an aligned language model's ability to refuse by projecting a single 'refusal direction' out of the weights that write th

safetyarxiv-cs-cl

It's All in the Way You Say It: The Role of Information Representation in LLM-Based Glycemic-Event Prediction

arXiv:2609.08772v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being investigated for physiological time-series prediction, yet their effectiveness may depend not only o

Multi-Functional Embedding Models for Funder Name Disambiguation in Scientific Publication Records

arXiv:2609.09984v1 Announce Type: new Abstract: Understanding the historical allocation and distribution of research funding advances our knowledge of how scientific research is supported across field

Agents

ResidualAuth: What Authorization State Must Language Agents Preserve under Revocable Delegation?

arXiv:2609.08062v1 Announce Type: new Abstract: Tool-using language agents can delegate and revoke permissions while acting through external services. We show that two authorization histories can have

agentsarxiv-cs-ai

SciLitBench: Benchmark and Design Principles for LLM-Powered Systematic Literature Reviews

arXiv:2609.05505v1 Announce Type: new Abstract: Systematic reviews require sustained human judgment across thousands of records, yet existing evaluations of large language models (LLMs) typically exam

SerenAI: State-transition system inspired by text-based world AI models

arXiv:2609.06647v1 Announce Type: new Abstract: Although professional workflows leverage large language models widely, the interpretation for auditing unconstrained free-text generation is usually int

Agents

The Internal Anatomy of Strategic Choice in Large Language Models

arXiv:2609.07478v1 Announce Type: new Abstract: Large language models act as strategic agents and models of human choice, yet choosing like a strategic agent does not mean computing like one. We recor

agentsarxiv-cs-ai

VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models

arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric