Search: “llm”

Which LLM is actually best at pentesting? benchmark to find out

Hey all, You’ve probably noticed that using LLMs and AI agents for pentesting has become pretty common lately. The problem is there isn’t really a good way to figure out which model is actually best s

30 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click

DGX agent

arXiv:2608.26867v1 Announce Type: new Abstract: Large language model agents reason, call tools, and act autonomously over many steps, but their agentic skills-correctly sequencing tools, planning unde

model-releasesarxiv-cs-ai

28 Aug 2026

HybridProver: Augmenting Theorem Proving with LLM-Driven Proof Synthesis and Refinement

arXiv:2505.15740v2 Announce Type: replace-cross Abstract: Formal methods play a crucial role in ensuring the reliability of critical systems through rigorous mathematical verification. However, their

MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering

model-releasesarxiv-cs-cl

arXiv:2601.15236v2 Announce Type: replace Abstract: Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a si

The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts

arXiv:2608.26235v1 Announce Type: new Abstract: Accuracy-only benchmarking of reasoning-capable large language models misses a central deployment question: when do extended thinking tokens earn their

Local Ai

What are the agentic softwares you run on top of local llm which can do tasks for you ?

What is your to software and why ? I am trying to learn first before I am pick a software. I already have Ollama and few models running which I can acces thru terminal and open web UI. My goal for now

local-air-ollama

A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks

model-releasesarxiv-cs-cl

arXiv:2608.26008v1 Announce Type: cross Abstract: Large language models (LLMs) remain vulnerable to jailbreak attacks that exploit techniques such as role-playing, obfuscation, code transformation, an

Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace

model-releasesarxiv-cs-cl

arXiv:2608.24958v1 Announce Type: cross Abstract: An audio language model is a black box in a specific way: we see what it says, never what it works out on the way there, and chain-of-thought monitori

Local LLM harness for reverse engineering software?

Does anyone know of a decent reverse engineering harness/workspace setup for local models that I can just point the model at and have it go to work until it's reversed most if not all of the functions

Agents

MacroAgent: Regularity-Aware Macro Legalization with LLM-Agent-Designed Contour Algorithms

arXiv:2608.24946v1 Announce Type: new Abstract: Macros constitute a large part of the core area in modern very large-scale integration (VLSI) designs. Moreover, macro positions have a significant impa

agentsarxiv-cs-lg

Agents

MTDiag: A Multi-Turn Diagnostic Dataset Towards Clinically Meaningful LLM Evaluation

arXiv:2608.25085v1 Announce Type: new Abstract: Clinical diagnosis is fundamentally interactive and incremental, yet the dominant paradigm for evaluating Large Language Models (LLMs) in medicine remai

agentsarxiv-cs-cl

Running local LLM's as agents in Claude Code

I hit my token limit three times a day on my max subscription - got sick of that and designed this MCP setup to shift some of the coding load to my local Qwen3.8-27B model. I've been iterating on it n

Agents

Confident at the moment of action: belief miscalibration in LLM play under hidden information

arXiv:2608.24691v1 Announce Type: new Abstract: Agentic systems increasingly gate actions on a model's own stated confidence, which assumes confidence tracks correctness at the moment of acting. We te

agentsarxiv-cs-ai

EMRB: A Multi-Level Benchmark for Evaluating LLM Reasoning over Raw Electromagnetic Signals

arXiv:2608.24086v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as code agents for scientific and engineering analysis, but their ability to analyze raw physical-lay

Has anyone tried running a distributed LLM across multiple phones?

So I've been going down a rabbit hole trying to run local LLMs without dropping 800+ on a Mac or GPU. RAM prices are insane right now and I realized that used phones have decent unified memory for dir

model-releasesr-ollama

How to Fine-Tune an LLM: An End-to-End Guide

I ended up fine tuning a mistral 7b to outperform our costly foundational model and saved $300k. I previously thought that fine tuning was pointless (it's definitely not) and that all these problems c

Learning from the Right Rollouts: Data Attribution for PPO-based LLM Post-Training

arXiv:2604.01597v2 Announce Type: replace Abstract: Traditional RL algorithms like Proximal Policy Optimization (PPO) typically train on the entire rollout buffer, operating under the assumption that

safetyarxiv-cs-lg

NeuronGuard: Robust LLM Safety Alignment via Ablation-Aware Safety Signal Redistribution

arXiv:2608.23959v1 Announce Type: cross Abstract: Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms thr

Quasar: A Programming Language Specialized for LLM Code Actions

arXiv:2506.12202v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often call external tools to solve tasks. One effective strategy is for LLMs to write code, enabling them to use

RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons

arXiv:2608.24758v1 Announce Type: new Abstract: Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-le

ReflCtrl: Controlling LLM Reflection Efficiently via Representation Engineering

arXiv:2512.13979v2 Announce Type: replace Abstract: Large reasoning models achieve strong performance on diverse tasks by producing extended chains of thought. Self-reflection, the ability to review a

Activation-Weighted Seeded Residual Coding for Low-Bit LLM Weight Repair

arXiv:2608.23144v1 Announce Type: cross Abstract: Low-bit weight quantization saves storage but leaves errors that degrade language-model quality. We introduce Activation-Weighted Seeded Residual Codi

Aligned Alone, Misaligned Together: Forecasting Adversarial Capture in LLM Agent Populations

arXiv:2608.22444v1 Announce Type: new Abstract: The unit of AI safety evaluation is still the individual model, yet language-model agents are increasingly deployed in interacting populations that read

safetyarxiv-cs-cl

Beyond the Stability-Exploration Dilemma: Environmental Regularization for LLM Policy Optimization

arXiv:2608.23311v1 Announce Type: new Abstract: Policy optimization (PO) for Large Language Models faces a stability--exploration trade-off, currently mediated by an action-side Policy-KL regularizer.

safetyarxiv-cs-cl

Bi-EZP: LLM-Guided Bilevel Program Evolution for Ensemble Zero-Cost Proxy Discovery

arXiv:2608.21927v1 Announce Type: cross Abstract: Zero-cost proxies enable neural architecture search (NAS) to rank candidate networks from statistics computed at initialization, avoiding repeated tra

ChemDIRT: A Diversified Instruction, Representation, and Task Benchmark for Robust Chemistry-LLM Evaluation

model-releasesarxiv-cs-lg

arXiv:2608.21504v1 Announce Type: new Abstract: The rapid advancement of large language models (LLMs) has led to increasing interest in their application to scientific domains such as chemistry. Howev

Closed-Loop Bayesian Molecular Inverse Design with Semantic LLM Surrogates

arXiv:2608.22967v1 Announce Type: new Abstract: Practical molecular inverse design is rarely a one-shot generation problem; it often takes the form of closed-loop candidate-pool enrichment, where unde

Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration

arXiv:2607.27836v2 Announce Type: replace Abstract: Large language model unlearning is consistently fragile under relearn attacks. On TOFU, fine-tuning on twenty forget examples substantially recovers

CyrillicQA: The Influence of Phonetically Encoded Secret Language on LLM Performance

arXiv:2608.21462v1 Announce Type: cross Abstract: Due to the selection of their training data, large language models (LLMs) perform best on standard-language inputs from languages using the Latin alph

Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code

arXiv:2608.22652v1 Announce Type: cross Abstract: LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into

Local Ai

Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits

arXiv:2607.16646v2 Announce Type: replace-cross Abstract: Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, and they fail si

local-aiarxiv-cs-ai

Hidden in the Request: Explaining Unethical LLM Compliance through Token Relevance

arXiv:2608.23264v1 Announce Type: new Abstract: Although Large Language Models (LLMs) are aligned to optimize for both helpfulness and harmlessness, these dual objectives may conflict, inevitably lead

Lexical Perturbations Disrupt LLM Reasoning: An Empirical Study of Attention Diversion

arXiv:2608.22140v1 Announce Type: cross Abstract: Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We

LLM-Based Selection of Incongruent Verbal and Nonverbal Behavior for Virtual Humans

arXiv:2608.22731v1 Announce Type: new Abstract: Nonverbal behavior generation systems for virtual agents often take an utterance as input and generate nonverbal behaviors that emphasize or illustrate

LLM for pentesting?

Hi, I’m having fun using “huihui_ai/Qwen3.6-abliterated:35b-Claude-4.7” for pen testing. It’s uncensored so it does whatever I want. I’m on M4 Pro 14 core 64GB. Just wondering if there are some models

model-releasesr-ollama

PsychJail: Exploring Psychological Jailbreaks via Multi-Turn Persuasion of LLM Policies

arXiv:2608.23028v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in education, healthcare, policy advising, and other interactive settings, where users engage the

There Is No Neutral Harness: Modern LLM Leaderboards Are Manufactured by Config-Fragile Items

arXiv:2608.21382v1 Announce Type: new Abstract: Multiple-choice benchmarks fix the questions and the correct answers, but not the harness: the order of the options, the wording of the prompt, and whet

Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation

arXiv:2608.22230v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for hate speech moderation, often within human--AI workflows in which reviewers provide feedback befo

Certified Multi-Turn Robustness for LLM Safety via Compositional Bounds and Safety Persistence

arXiv:2608.20820v1 Announce Type: new Abstract: Large language models (LLMs) are vulnerable to multi-turn jailbreak attacks that progressively manipulate conversation context. Existing certified robus

I developed my own quantized LLM from scratch, trained on 30B tokens, deploys in 60 MB

Reposting here after sharing this on [r/MachineLearning](r/MachineLearning) a few days ago, where it got a much better response than I expected (300+ upvotes, great questions, zero roasting) GitHub is

Knowing but Not Saying: Preventing Factual Access Failures in LLM SFT via Recall-Anchored Distillation

arXiv:2608.20794v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) can degrade factual behavior outside the target domain. This degradation is often described as catastrophic forgetting, yet

Recognition-Conditioned Reasoning: A Training-Free Multimodal-LLM Pipeline for Fine-Grained Micro-Action Understanding

arXiv:2608.21022v1 Announce Type: new Abstract: Micro-actions are subtle, short, low-amplitude body movements, such as a fidgeting hand or a slight head tilt, that humans perform with little conscious

researcharxiv-cs-cv

Specification Portability Across LLM Development Agents: Cross-Agent Compatibility in Specification-Driven Software Migration

arXiv:2608.21208v1 Announce Type: cross Abstract: This paper investigates cross-agent specification portability using Oracle-to-PostgreSQL migration as a controlled software transformation task. The s

UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialists

arXiv:2608.20918v1 Announce Type: new Abstract: Organizations maintain task-specific adapters for open-weight language models, and each new base-model release forces a migration decision: retain exist

When the Feature Pool Goes Algorithmic: Extending Mufwene's Ecology of Language Evolution to LLM-Mediated Exposure

arXiv:2608.21088v1 Announce Type: new Abstract: Mufwene's ecological model locates language evolution in competition among variants contributed by individual idiolects and in speakers' selection from

Computational Phenomenology of Borderline Personality Disorder: A Comparative Evaluation of LLM-Simulated Expert Personas and Human Clinical Experts

arXiv:2508.19008v3 Announce Type: replace Abstract: Building on a human-led thematic analysis of clinical life-story interviews (> 150,000 words) with inpatients with Borderline Personality Disorder,

21 Aug 2026

InsufficiencyBench: Evaluating LLM legal advice on underspecified user queries

arXiv:2608.20220v1 Announce Type: new Abstract: Legal AI systems are increasingly used to answer legal questions, yet existing benchmarks assume queries arrive fully specified. In practice, users omit

21 Aug 2026

Mitigating Identity Essentialism in LLM Agents with Longitudinal Life Trajectories