Search: “evaluation”

okay okay, hillclimbing agent in the background. love it. great job @hwchase17 @Vtrivedy10 and team. 🚀Launching: LangSmith Engine LangSmith Engine is an agent that sits on top of your traces It runs i

the langsmith team cooked on this one @jakebroekhuizen this + claude code + langsmith cli has been 🔥 🚀Launching: LangSmith Engine LangSmith Engine is an agent that sits on top of your traces It runs i

Sebastian Raschka shared an HTML-formatted table comparing active parameter ratios across different large language model architectures for improved readability and to avoid text truncation. The resour

Agent observability is more than just logging and viewing traces. You need agents sitting on top of your traces to proactively identify issues. We're launching Engine. Think of it as an always-on AI e

Has been a blast shipping Engine!! We built engine to automate the agent development loop, and make agents that improve autonomously. Absolutely cracked team shipping at light speed to create the futu

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kin

Langsmith Engine appears to be a seriously high value release from the @LangChain team A deep agent that’s monitoring Langsmith agent traces and not only surfaces issues but: - proposes a PR to resolv

🚀Launching: LangSmith Engine LangSmith Engine is an agent that sits on top of your traces It runs in the background and automatically identifies issues It then proactively suggests action items (code

Observability going from 'read the traces' to 'the traces read themselves and tell you what to fix.' That's the feedback loop most agent teams are missing. 🚀Launching: LangSmith Engine LangSmith Engin

The UK AISI found Mythos Preview is the first model to solve both their cyber ranges end-to-end. No model had ever solved the AISI’s “Cooling Tower” cyber range before. We're getting it to defenders a

This looks great - kudos @LangChain team 🚀Launching: LangSmith Engine LangSmith Engine is an agent that sits on top of your traces It runs in the background and automatically identifies issues It then

arXiv:2605.09852v1 Announce Type: new Abstract: Machine learning algorithms are being used in high-stakes decisions, including those in criminal justice, healthcare, credit, and employment. The resear

$NBIS announced a partnership with LangChain to integrate Nebius Token Factory with LangChain’s Deep Agents. The goal is to make it easier for teams building AI agents on LangChain to run those worklo

NEW paper from Google DeepMind. (bookmark it) AI Co-Mathematician is an agentic research workbench for mathematicians, and it just hit 48% on FrontierMath Tier 4, a new high score among AI systems eva

thoughts after doing a bunch of synthetic data gen for eval + environment building - LLMs are incredible projections of the world bundled into a set of weights - but doing targeted extraction of certa

// LLMs Improving LLMs // Interesting progress the past of couple of weeks around self-improving AI agents. If autoresearch was interesting, you will like this read. (bookmark it) We've been hand-tuni

Gary Marcus analyzes and critiques Geoffrey Hinton's arguments regarding weaknesses in deep learning and artificial neural networks. The article likely examines specific technical or conceptual claims

Hot take on METR’s new graph that so many people are flipping about today. • Claude Code is a real advance; Mythos probably builds on some of what is learned there. But… • If you read the graph carefu

This post from Harrison Chase discusses how AI agents have evolved beyond just the underlying model, emphasizing critical infrastructure components including state management, observability, retry mec

When I started to work on the alignment problem more than 10 years ago, we had no idea how AGI was going to be built or how to make it safe. The field had maybe a dozen people who were working on it a

.@huggingface's agentic robotics app store for Reachy Mini is a big step toward more accessible physical AI. 🙌 Excited to see NVIDIA Isaac GR00T N integrated with Hugging Face LeRobot, helping develop

Harrison Chase expresses support for expanding AI benchmarking efforts and highlights appreciation for work being done by Harvey (likely Harvey AI, a legal AI company). The post indicates enthusiasm f

Gary Marcus reflects on changing public perception of Sam Altman, noting that skepticism toward Altman's claims was once dismissed as hatred but has become more mainstream. The post suggests a shift i

Claude Opus 4.7 just implemented an AlphaZero-style self-play pipeline from scratch. It did this on consumer hardware in three hours, then beat the Pascal Pons solver 7 of 8 as first-mover on Connect

NEW paper from Microsoft Research. If you care about training computer-use agents, this is one to keep. (bookmark it) The team builds 1,000 synthetic computers (each with realistic directory structure

AgentTrove is a newly released agentic dataset containing 1.7 million samples, designed to address the shortage of agentic datasets available on Hugging Face Hub. The dataset aims to support training

arXiv:2604.26269v1 Announce Type: cross Abstract: The essence of good creative writing is calibrated surprise: when constraints from all relevant dimensions act together, the feasible solution space c

Hugging Face introduced a new CLI command that helps users identify the best open-source models suited for their specific datasets, streamlining the model selection process. This feature reduces the m

Today we’re releasing Qwen-Scope 🔭, an open suite of sparse autoencoders for the Qwen model family. It turns SAE features into practical tools： 🎯 Inference — Steer model outputs by directly manipulati

Most pharma agent pilots never make it to production. We’re partnering with @Axtria to close this gap. Built on LangSmith, Axtria’s pharma-native AgentOps framework will give life sciences enterprises

Gary Marcus critiques the rapid deployment of chatbots to market without adequate consideration of potential harms and consequences. The statement reflects concerns about insufficient safety testing a

Xiaomi MiMo-V2.5-Pro achieves multiple breakthroughs in the latest Arena rankings (Apr 26, 2026) 🔥 🏆 Text Arena (Expert) — #6 globally | #1 open-source model Also #1 among Chinese models, with Xiaomi

I post-trained Qwen3-Coder to fix bugs using an actual debugger. The result: Solve rate: 70% → 89% Median turns to fix: 46 → 19 (-59%) Instead of just reading code or print-debugging, it: - reasons fr

OpenAI released a new dataset on Hugging Face Hub designed to improve ChatGPT's performance in clinical settings, addressing the specific needs and workflows of healthcare professionals. The dataset s

Comprehensive Multilingual Text Rendering： Better glyph accuracy, more consistent typography, and cleaner layouts even in complex compositions. It handles mixed-language scenarios more gracefully as w

Every time I tell AI utopianists that biology is too complex for AI to 'solve', they cite the success of AlphaFold. No, AlphaFold did not 'solve' protein folding. It gets broad structures correct ~70-

We built HF for AI builders collaboration, fun to see it's increasingly becoming the place for agent collaboration! This morning I'm sending my ml-intern to participate in the @OpenAI Parameter Golf c

What if instead of building one giant AI, we evolved a coordinator to orchestrate a diverse team of specialized AIs? 🐟 Excited to share our new paper: “TRINITY: An Evolved LLM Coordinator”, published

DeepSeek released a new open-weights model with strong benchmark performance, though Mollick notes that benchmark results may not fully capture the capabilities of open models compared to closed syste

LangSmith for Startups Spotlight: @AdaptiveBuilds Adaptive is the AI-native project accounting platform built for construction. They embed AI agents directly into job costing, billing, and forecasting

Cool paper on diversity collapse in AI agents. It's a common issue with all the deployed multi-agent systems. New paper shows that multi-agent LLM systems converge on near-identical outputs over time,

GPT-5.5 achieved state-of-the-art performance on the ARC-AGI-2 benchmark, with scores ranging from 85.0% on maximum difficulty tasks to 33% on low difficulty tasks. The model demonstrated consistent i

If you want to stack rank LLMs/VLMs on document understanding 📄, you can through ParseBench, now live on @kaggle 📊 ParseBench is the most comprehensive document OCR benchmark over real enterprise docu

Interesting, OpenAI just released a free healthcare version of ChatGPT-5.4 for clinicians that beat specialty-matched physicians with unlimited time + web access on a benchmark of real & hard clinical

arXiv:2602.11675v3 Announce Type: replace Abstract: Large language models may answer causal questions correctly for the wrong reasons, substituting associational shortcuts P(Y|X) for the interventiona

Rivian and Volkswagen Group's joint venture put Devin to work on testing and ticket triage across a software platform that will power up to 30M vehicles. Devin handles autonomous ticket triage in Slac