GLM 5.3 SlopCodeBench Results
Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20
Knowledge catalogue
Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20
arXiv:2608.17205v1 Announce Type: new Abstract: Vision-language models (VLMs) combine images and text, but when the two conflict and one becomes harder to read, it is unclear how a model shifts its re
arXiv:2608.15428v1 Announce Type: cross Abstract: Multiple-choice benchmarks are graded on whether a model picks the right option, not on whether it needed the question. Measuring that gap takes care:
Powered by @Cerebras, Ultrafast generates up to 750 tokens per second, bringing our most intelligent model to products and workflows where every second counts. Ultrafast is designed for businesses whe
arXiv:2608.11469v1 Announce Type: cross Abstract: AI agents are rapidly improving in cybersecurity capabilities when the source code is available for analysis, yet much of the software most consequent
arXiv:2608.10764v1 Announce Type: new Abstract: Counterfactual video understanding evaluates whether models grasp physical and commonsense regularities. However, existing multiple-choice question (MCQ
arXiv:2608.07584v1 Announce Type: new Abstract: Vision-language models (VLMs) have made rapid progress in visual perception and increasingly support real-world tasks that depend on images. Many such t
arXiv:2608.03070v1 Announce Type: cross Abstract: Frontier AI model developers increasingly rely on layered safeguards to prevent catastrophic misuse, but little public evidence exists on how much pro
DeepSeek V4 Flash: Preview → 2026-07-31 Benchmark Preview 0731 Δ Terminal Bench* 56.9 82.7 +25.8 Toolathlon 51.8 70.3 +18.5 NL2Repo — 54.2 new Cybergym — 76.7 new DeepSWE — 54.4 new Agent Last Exam —
Artificial Analysis: DeepSeek V4 Flash scores 50 on the Artificial Analysis Intelligence Index, matching Gemini 3.6 Flash and up 10 points from the preview launch in April — DeepSeek V4 Flash 0731 is
Amazon Bedrock Advanced Prompt Optimization optimizes your prompts for up to 5 models at once and compares original versus optimized performance across quality, latency, and cost. Migrate to a new mod
arXiv:2607.23332v1 Announce Type: new Abstract: Creating a reusable tool is an investment: an agent pays a fixed cost now in exchange for the potential of future reuse. Therefore, a user should prefer
On FrontierCode 1.1, our benchmark for real-world engineering tasks that grades mergeability and quality, Opus 5 scores 63.6% with a 69.6% pass rate on Extended — approaching Fable 5 at half the cost.
Are AI labs pelicanmaxxing? Excellent piece of work by Dylan Castillo, who took a deep-dive into the frequently pondered question of whether the AI labs have been deliberately training models to draw
Earlier this month I hosted a fireside chat session at the AI Engineer World's Fair with Cat Wu and Thariq Shihipar from Anthropic's Claude Code team. We talked about Claude Code, Claude Tag, Fable, c
simonw/pedalican Clearly I wasn't paying attention when these were first announced back in May, but today I accidentally activated a 'pet' in Codex Desktop - a little animated robot, reminiscent of Cl
Here is my other heavily used pattern. Evaluator/Judge: Fable 5 Executor: GPT-5.5 I no longer wait for frontier models or am loyal to any. I now spend more time on better orchestration, harness, skill
Learn how to run OpenAI GPT-6 Sol, Luna, and Astra on Amazon Bedrock through global cross-Region inference from Singapore, Jakarta, Thailand, Malaysia, and Taipei. This post covers the Converse and Op
arXiv:2610.13098v1 Announce Type: new Abstract: Visual agents for symbolic graphics reasoning depend on executable world representations, yet continued improvement remains bottlenecked by ambiguous su
AI solved open mathematical problems. But the biggest surprise came afterwards. A team at Google DeepMind has used AI to resolve 9 open Erdős problems, including questions that had remained unsolved f
Banger paper from Meta Superintelligence Labs on self-improving agents. (bookmark it) It's hard to know exactly what drives self-improvement, since so many variables are at play (notes, skills, tool c
arXiv:2610.11901v1 Announce Type: new Abstract: Stance detection requires identifying an author's attitude toward a given target, sometimes based on conversational context. Jev, a specialized decision
arXiv:2610.10809v1 Announce Type: new Abstract: Instance-specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale,
arXiv:2610.10827v1 Announce Type: cross Abstract: Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate int
arXiv:2610.12233v1 Announce Type: cross Abstract: Recursive self-improvement, the participation of AI systems in improving their own capabilities, is beginning to move from theoretical prospect to pra
Aisha Malik / TechCrunch: AI hardware startup Natura unveils Interface, a 99 smart ring that connects to AI agents and apps, including Claude, ChatGPT, Muse, Grok Bot, and Instinct — Meet Interface, a
arXiv:2610.10163v1 Announce Type: new Abstract: Linking people's appearance and actions to character identities is essential for understanding video narratives. We present a framework for identity-awa
arXiv:2609.32701v2 Announce Type: replace-cross Abstract: In security-sensitive applications, language-model agents are often required to coordinate without disclosing confidential information. Yet re
arXiv:2610.09079v1 Announce Type: cross Abstract: Large language models equipped with development environments have moved code generation toward repository-scale construction, yet building complete re
Lunara introduces a novel Diffusion Mixture Transformer architecture with fewer than 10B active parameters for modeling artistic intelligence in image generation. Its CAT training algorithm iterativel
arXiv:2610.09458v1 Announce Type: new Abstract: When an LLM answers a state-specific policy question wrongly, it may be hallucinating, or it may be returning a real value that holds in another state.
Ultrafast is rolling out today for GPT-6.1 Sol in the API, Codex, and ChatGPT Work. Near-Astra intelligence at up to 8x faster speeds than Sol Standard, so you can build as fast as the ideas come. Med
arXiv:2610.08660v1 Announce Type: new Abstract: Background: Biomedical AI can generate plausible explanations without reliably verifying whether each statement is supported by patient-specific evidenc
arXiv:2610.07832v1 Announce Type: cross Abstract: Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, ex
arXiv:2610.08246v1 Announce Type: new Abstract: Frontier large language models (LLMs) can generate heuristic functions that guide search to achieve state-of-the-art performance in satisficing planning
arXiv:2610.07792v1 Announce Type: cross Abstract: Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correct
arXiv:2610.08720v1 Announce Type: new Abstract: LLM-based agents are increasingly advancing scientific and engineering problem solving, with physics simulation emerging as a challenging yet practical
arXiv:2610.04976v1 Announce Type: new Abstract: We evaluate language models' capability to generate novel research ideas through the lens of compression. We aim to lower-bound the potentially tiny pro
We've been building a benchmark out of real concurrency bugs (race conditions, deadlocks, cancellation issues) taken from merged PRs in about 100 Python projects. Each task gets graded by the project'
arXiv:2610.03226v1 Announce Type: cross Abstract: GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal ho
arXiv:2610.03153v1 Announce Type: cross Abstract: Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resources
arXiv:2610.03675v1 Announce Type: cross Abstract: LLM-guided evolutionary methods, such as AlphaEvolve, have emerged as powerful approaches for challenging computational optimization problems, such as
arXiv:2610.03695v1 Announce Type: new Abstract: Modern chess engines are silent experts: they play at a superhuman level, but do not offer explanations for their play. On the other hand, language mode
arXiv:2610.02788v1 Announce Type: new Abstract: Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embod
Banger paper from NVIDIA on test-time compute for terminal agents. The finding is that you should sample several candidate shell commands, verify them before running one, and spend more on the verifie
'create the ultimate brutalist city builder. you should have lots of controls over buildings in ways that are delightful and intuitive to people...' Here is Fable 5.1, a little over a year since GPT-5
Hey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its brows
arXiv:2610.00376v1 Announce Type: new Abstract: We evaluate Jev on ten dataset-defined application labels in CESNET-QUICEXT-25 using only the first ten packets' sizes, directions, and inter-packet tim
arXiv:2610.00885v1 Announce Type: cross Abstract: Coding agents increasingly automate Lean proof development, but successful compilation alone does not establish that a candidate proves the intended s
arXiv:2610.01079v1 Announce Type: cross Abstract: Machine-learning Network Intrusion Detection Systems (IDS) depend on substantial labeled datasets and task-specific training, whereas Large Language M
arXiv:2610.00834v1 Announce Type: new Abstract: ARC-AGI-3 evaluates agents in interactive environments whose rules and objectives must be inferred from observation. We present Kepler, an open-source h
arXiv:2609.39238v1 Announce Type: new Abstract: An agent that moves must recognise a place from a viewpoint it has never seen. We introduce 4MT-VLM, a dataset of procedurally generated landscapes, eac
arXiv:2609.38737v1 Announce Type: new Abstract: Desk-based learning and creative activities benefit from handwritten engagement. However, current generative AI tools deliver guidance through a separat
arXiv:2609.38891v1 Announce Type: new Abstract: Long-horizon agentic tasks demand strong reasoning and efficient execution across successive interactions with dynamic environments. A common approach d
arXiv:2609.38948v1 Announce Type: cross Abstract: Frontier models excel at many digital benchmarks, yet their ability to drive a real car, an everyday human skill, remains largely untested. We present
arXiv:2609.38415v1 Announce Type: cross Abstract: This technical report presents an alignment evaluation developed and performed by the UK AI Security Institute for assessing whether advanced AI syste
arXiv:2609.40340v1 Announce Type: new Abstract: Evolutionary search with large language models (LLMs) can stall when progress requires external knowledge the model lacks. Supplying relevant documents
arXiv:2609.38181v1 Announce Type: new Abstract: Survival analysis estimates time-to-event outcomes from patient covariates and is widely used for medical risk assessment. Patients seeking prognostic i
arXiv:2609.39982v1 Announce Type: cross Abstract: Terminal agents act through stochastic model generations, yet the ability to generate a useful action does not ensure its reliable execution. A poor c
arXiv:2609.32754v2 Announce Type: replace Abstract: Large language model agents can often make reasonable local decisions on short tasks, yet their performance degrades when success requires long sequ