Search: “gpt”

GeoRC: A Benchmark for Geolocation Reasoning Chains

arXiv:2601.21278v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) are good at recognizing the global location of a photograph -- their geolocation prediction accuracy rivals the

Tutorials

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?

DGX agent

arXiv:2604.17930v1 Announce Type: new Abstract: Large Language Models (LLMs) exhibit a puzzling disparity in their formal linguistic competence: while they learn some linguistic phenomena with near-pe

tutorialsarxiv-cs-cl

21 Apr 2026

Safety

High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning

arXiv:2507.05920v2 Announce Type: replace Abstract: State-of-the-art large multi-modal models (LMMs) face challenges when processing high-resolution images, as these inputs are converted into enormous

safetyarxiv-cs-cv

How Robustly do LLMs Understand Execution Semantics?

arXiv:2604.16320v1 Announce Type: cross Abstract: LLMs demonstrate remarkable reasoning capabilities, yet whether they utilize internal world models or rely on sophisticated pattern matching remains o

Research

HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models

arXiv:2511.01066v3 Announce Type: replace Abstract: We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 tr

researcharxiv-cs-cl

Tools

I for one would be delighted to see OpenAI commit to maintaining a tool like this in the long-term, I'm already nervous about mine going sta…

Simon Willison expresses hope that OpenAI will commit to long-term maintenance of an AI tool, citing concerns about his own tool potentially becoming stale or discontinued. The post reflects broader u

toolssimon-willison--x

Kimi K2.6 autonomously overhauled exchange-core, an 8-year-old open-source financial matching engine. Over a 13-hour execution, the model it…

model-releaseskimi-moonshot--x

Kimi K2.6 autonomously overhauled exchange-core, an 8-year-old open-source financial matching engine. Over a 13-hour execution, the model iterated through 12 optimization strategies, initiating over 1

Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection

arXiv:2506.00955v2 Announce Type: replace Abstract: Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity. In addition, exi

LLMs are still not consistent judges of qualitative work, and small changes to how that work is presented affect outcomes. Better harnessing…

model-releasesethan-mollick--x

LLMs are still not consistent judges of qualitative work, and small changes to how that work is presented affect outcomes. Better harnessing and methods (multiple judging runs with randomized orders,

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training

arXiv:2509.11983v2 Announce Type: replace Abstract: Neural network (NN) training is inherently a large-scale matrix optimization problem, yet the matrix structure of NN parameters has long been overlo

MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

arXiv:2604.18584v1 Announce Type: cross Abstract: Mathematical problem solving remains a challenging test of reasoning for large language and multimodal models, yet existing benchmarks are limited in

Modeling Multi-Dimensional Cognitive States in Large Language Models under Cognitive Crowding

arXiv:2604.17174v1 Announce Type: new Abstract: Modeling human cognitive states is essential for advanced artificial intelligence. Existing Large Language Models (LLMs) mainly address isolated tasks s

NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions

arXiv:2604.16493v1 Announce Type: cross Abstract: Natural Language to SQL (NL2SQL) technology empowers non-expert users to query relational databases without requiring SQL expertise. While large langu

Tools

OpenAI's new Euphony tool works almost exactly the same way as my Codex transcript viewer https://tools.simonwillison.net/codex-timeline?url…

OpenAI's new Euphony tool works almost exactly the same way as my Codex transcript viewer https://tools.simonwillison.net/codex-timeline?url=https%3A%2F%2Fgist.githubusercontent.com%2Fsimonw%2Fa9eb599

toolssimon-willison--x

Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

model-releasesarxiv-cs-cv

arXiv:2604.16515v1 Announce Type: new Abstract: The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their a

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

arXiv:2604.17338v1 Announce Type: cross Abstract: Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but o

Program Structure-aware Language Models: Targeted Software Testing beyond Textual Semantics

arXiv:2604.17715v1 Announce Type: cross Abstract: Recent advances in large language models for test case generation have improved branch coverage via prompt-engineered mutations. However, they still l

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

arXiv:2603.05863v2 Announce Type: replace Abstract: While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward

RoMathExam: A Longitudinal Dataset of Romanian Math Exams (1895-2025) with a Seven-Decade Core (1957-2025)

arXiv:2604.16392v1 Announce Type: cross Abstract: AI in Education research increasingly relies on authentic, curriculum-grounded assessment data, yet large, well-structured exam corpora remain scarce

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows

arXiv:2505.19897v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have extended their impact beyond Natural Language Processing, substantially fostering the development of interdi

SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translation

arXiv:2604.16511v1 Announce Type: cross Abstract: We present SQL Query Engine, an open-source, self-hosted service that translates natural language questions into validated PostgreSQL queries through

ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection

arXiv:2508.11281v3 Announce Type: replace Abstract: Detecting toxic content using language models is crucial yet challenging. While substantial progress has been made in English, toxicity detection in

Research

When Informal Text Breaks NLI: Tokenization Failure, Distribution Shift, and Targeted Mitigations

arXiv:2604.16787v1 Announce Type: new Abstract: We study how informal surface forms degrade NLI accuracy in ELECTRA-small (14M) and RoBERTa-large (355M) across four transforms applied to SNLI and Mult

researcharxiv-cs-cl

XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants

arXiv:2503.14281v4 Announce Type: replace-cross Abstract: AI coding assistants are widely used for tasks like code generation. These tools now require large and complex contexts, automatically sourced

Accelerate Generative AI Inference on Amazon SageMaker AI with G7e Instances

model-releasesaws-ml-blog

Today, we are thrilled to announce the availability of G7e instances powered by NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs on Amazon SageMaker AI. You can provision nodes with 1, 2, 4, and 8 RT

Characterising LLM-Generated Competency Questions: a Cross-Domain Empirical Study using Open and Closed Models

arXiv:2604.16258v1 Announce Type: new Abstract: Competency Questions (CQs) are a cornerstone of requirement elicitation in ontology engineering. CQs represent requirements as a set of natural language

COEVO: Co-Evolutionary Framework for Joint Functional Correctness and PPA Optimization in LLM-Based RTL Generation

arXiv:2604.15001v2 Announce Type: replace Abstract: LLM-based RTL code generation methods increasingly target both functional correctness and PPA quality, yet existing approaches universally decouple

Research

Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles

arXiv:2504.04927v2 Announce Type: replace-cross Abstract: As generative AI (GenAI) is increasingly applied in persona development to represent real users, understanding the implications and limitation

researcharxiv-cs-cl

Applications

Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning

arXiv:2604.16029v1 Announce Type: new Abstract: Parallel reasoning enhances Large Reasoning Models (LRMs) but incurs prohibitive costs due to futile paths caused by early errors. To mitigate this, pat

applicationsarxiv-cs-cl

Applications

Designing Synthetic Discussion Generation Systems: A Case Study for Online Facilitation

arXiv:2503.16505v4 Announce Type: replace-cross Abstract: A critical challenge in social science research is the high cost associated with experiments involving human participants. We identify Synthet

applicationsarxiv-cs-cl

DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

arXiv:2604.15866v1 Announce Type: cross Abstract: Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their ge

Facial-Expression-Aware Prompting for Empathetic LLM Tutoring

arXiv:2604.15336v1 Announce Type: cross Abstract: Large language models (LLMs) enable increasingly capable tutoring-style conversational agents, yet effective tutoring requires sensitivity to learners

From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text

arXiv:2604.16270v1 Announce Type: cross Abstract: The complexity of Vietnam's legal texts presents a significant barrier to public access to justice. While Large Language Models offer a promising solu

Tools

Important to note: that 3x increase for images is entirely due to Opus 4.7 being able to handle higher resolutions. I tried that again with …

Important to note: that 3x increase for images is entirely due to Opus 4.7 being able to handle higher resolutions. I tried that again with a 682x318 pixel image and it took 314 tokens with Opus 4.7 a

toolssimon-willison--x

LaMSUM: Amplifying Voices Against Harassment through LLM Guided Extractive Summarization of User Incident Reports

arXiv:2406.15809v5 Announce Type: replace Abstract: Citizen reporting platforms help the public and authorities stay informed about sexual harassment incidents. However, the high volume of data shared

LinuxArena: A Control Setting for AI Agents in Live Production Software Environments

arXiv:2604.15384v1 Announce Type: cross Abstract: We introduce LinuxArena, a control setting in which agents operate directly on live, multi-service production environments. LinuxArena contains 20 env

LLMs Corrupt Your Documents When You Delegate

arXiv:2604.15597v1 Announce Type: new Abstract: Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding

neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing

model-releasesarxiv-cs-cv

arXiv:2604.16170v1 Announce Type: new Abstract: We introduce neuralCAD-Edit, the first benchmark for editing 3D CAD models collected from expert CAD engineers. Instead of text conditioning as in prior

Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation

arXiv:2604.15937v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed to curate and rank human-created content, yet the nature and structure of their biases in these

QuantSightBench: Evaluating LLM Quantitative Forecasting with Prediction Intervals

arXiv:2604.15859v1 Announce Type: cross Abstract: Forecasting has become a natural benchmark for reasoning under uncertainty. Yet existing evaluations of large language models remain limited to judgme

SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systems

arXiv:2604.16022v1 Announce Type: new Abstract: As Large Language Models (LLMs) transition from text processors to autonomous agents, evaluating their social reasoning in embodied multi-agent settings

The Jensen + @dwarkesh_sp podcast was fantastic. Jensen is someone who understood how ecosystems work and someone who understands real-world…

model-releasessoumith-chintala--x

The Jensen + @dwarkesh_sp podcast was fantastic. Jensen is someone who understood how ecosystems work and someone who understands real-world trade, policy and controls work. And in some deeper sense h

The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring

arXiv:2604.15702v1 Announce Type: new Abstract: We introduce a cross-domain behavioural assay of monitoring-control coupling in LLMs, grounded in the Nelson and Narens (1990) metacognitive framework a

I think Artificial Analysis does a good job overall and provides transparency in benchmarking, but GDPval-AA is not a good benchmark and nee…

model-releasesethan-mollick--x

I think Artificial Analysis does a good job overall and provides transparency in benchmarking, but GDPval-AA is not a good benchmark and needs to stop being reported. It is Gemini 3.1 judging other mo

18 Apr 2026

ADAPT: Benchmarking Commonsense Planning under Unspecified Affordance Constraints

arXiv:2604.14902v1 Announce Type: cross Abstract: Intelligent embodied agents should not simply follow instructions, as real-world environments often involve unexpected conditions and exceptions. Howe

Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimization

arXiv:2604.14853v1 Announce Type: new Abstract: Test-time compute scaling, the practice of spending extra computation during inference via repeated sampling, search, or extended reasoning, has become

Anonpsy: A Graph-Based Framework for Structure-Preserving De-identification of Psychiatric Narratives

arXiv:2601.13503v2 Announce Type: replace Abstract: Psychiatric narratives encode patient identity not only through explicit identifiers but also through idiosyncratic life events embedded in their cl

AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models

arXiv:2505.10846v3 Announce Type: replace Abstract: This paper presents AutoRAN, the first framework to automate the hijacking of internal safety reasoning in large reasoning models (LRMs). At its cor