Search: “llm”

TypeSafe’s Jev classifies, scores, and routes without generating text — up to hundreds of times cheaper than an LLM judge. What that changes for evals, confidence routing, and application architecture

TensorRT Edge‑LLM ran Qwen 3.6‑27B on a single NVIDIA Jetson AGX Thor and completed the MLPerf Edge Agentic benchmark in 24 min 36 s—52.33 tokens/s—which is 6.4× faster than the llama.cpp reference (2

Learn how Azure AI Speech LLM 2607 improves multilingual and mixed-language recognition, lowers latency, and simplifies domain-specific customization with enhanced phrase lists. The post Announcing Az

OWASP GenAI Security Project Releases 2026 Top 10 for LLM Applications, Debuts Agent Control Standard and New Resources for Securing Generative and Agentic AI F5, WitnessAI, Evoke Security and Mondoo

OpenAI has built an LLM super-hacker called GPT-Red that it uses as a sparring partner to help its other models boost their defenses against cyberattacks. Last week the company released the latest ver

If you're struggling to get started with evals, you're not alone. This post explains how to write LLM evals as ordinary tests in CI with Phoenix, pytest, and Vitest/Jest. The post Evals in CI: How to

In this post, we show you how PAR built a production-ready multi-tenant LLM analytics system that enforces row-level security through a three-layer architecture: cryptographic request signing with AWS

Release: llm-anthropic 0.25.1 New model: Claude Opus 4.8 (claude-opus-4.8). New -o fast 1 option for fast mode, for organizations with that feature enabled on their account. Default max_tokens for eac

NVIDIA's Blackwell GB200 NVL72 architecture achieved the fastest-ever results on the STAC-AI benchmark for financial LLM inference, delivering up to 3.2x single-GPU performance improvements over the p

Databases have introduced new AI-powered SQL functions which take natural language instructions as input and are evaluated using LLMs. They leverage the power of LLMs to answer new kinds of queries: W

Release: datasette-llm 0.1a7 Mechanism for configuring default options for specific models. Part of Datasette's evolving support mechanism for plugins that use LLMs. It's now possible to configure a m

Release: llm-anthropic 0.25 New model: claude-opus-4.7, which supports thinking_effort: xhigh. #66 New thinking_display and thinking_adaptive boolean options. thinking_display summarized output is cur

Load balancing and scaling LLM serving involves distributing inference requests across multiple model instances or GPUs to prevent bottlenecks and ensure consistent response times under varying traffi

**llm-mistral 0.16** is a language‑model plugin that provides programmatic access to Mistral models via the official Mistral API. It exposes wrapper functions for sending requests, receiving responses

MLPerf Training v6.1 adds an agentic reinforcement-learning workload that measures how quickly systems can teach a 397-billion-parameter language model to repair real software projects. The post MLPer

We benchmarked Jev against Claude Opus 5 and GPT-5.6 Terra on accuracy, cost, and latency. Learn how threshold tuning changes hallucination detection. The post Jev vs. LLM-as-a-Judge: Accuracy and cos

Yael Grauer / The Verge: Anthropic and other researchers detail how thousands of people were catfished by dating scam apps using LLM-generated replies from Claude and other models — Security researche

Tiffany Hsu / New York Times: MIT launches the LLM Election Observatory, a dashboard tracking how nearly a dozen AI models tailor responses to political queries during the 2026 US midterms — Artificia

Moving Large Language Models (LLMs) from experimental prototypes into enterprise production exposes a critical truth: your infrastructure dictates both your performance ceilings and your unit economic

Jennifer Pattison Tuohy / The Verge: Anker unveils the Eufy MindBase, a local AI hub for its cameras that runs an on-device, Anker-developed LLM, with a dedicated AI chip and up to 48TB of storage — T

Speculative decoding speeds LLM inference by letting a lightweight draft model propose multiple tokens that the larger target model verifies in parallel, reducing decoding iterations while preserving

BenchMIRT is a new method for auditing LLM benchmarks question by question, revealing which capabilities they actually measure and helping researchers build smaller, more focused, and easier-to-interp

Chris Welch / Bloomberg: Sonos opens its platform to third-party AI assistants, upgrades Sonos 27voice assistant with an in-house LLM, and plans to let users create their own AI agents — As the compan

OpenAI’s LLM agents were trained to win a benchmarking competition and were given “impossible tasks” on the internal ExploitGym framework. With safety guardrails disabled, the agents created an improv

Shadow Engine Recovery in NVIDIA Dynamo keeps a fully-initialized standby LLM engine on the same GPU, using the GPU Memory Service (GMS) to persist model weights and share them across engines without

Reuters: Sources: Apple trained a China-specific LLM with Alibaba's support, which would make Apple the first foreign company to offer a proprietary AI model in China — Apple (AAPL.O) has trained a la

Anthony Ha / TechCrunch: YouTuber Hank Green faces online criticism after using ChatGPT to help research a script, and says his LLM usage “is not healthy for me or good for the world” — Hank Green, a

AI productivity is best measured by connecting AI usage to validated downstream outcomes. Tokens, prompts, and generated lines show activity, but they do not prove value. A better measurement model tr

Nonuniform Tensor Parallelism (NTP) enables large-scale LLM training jobs to dynamically adapt tensor parallelism degree in response to transient GPU unavailability, ensuring sustained Goodput and min

In this post, you will learn five practical patterns for building resilient generative AI applications on AWS, progressing from native Amazon Bedrock features to multi-model orchestration using an LLM

OpenAI and Broadcom unveiled Jalapeño, OpenAI's first Intelligence Processor designed specifically for LLM inference , with early testing showing substantially better performance per watt than current

OpenAI: OpenAI and Broadcom unveil Jalapeño, an LLM-optimized inference chip developed from design to manufacturing tape-out in nine months, aided by OpenAI's models — Loading... - Early testing shows

Dean W. Ball / @deanwball: Anthropic secretly limiting Claude's usefulness for LLM development strengthens the argument that Anthropic is using AI safety to justify monopolistic behavior — My last obs

Anyscale has introduced a new Agent Skill designed to enhance LLM post-training capabilities, likely enabling developers to build and train agentic systems more effectively using Ray's distributed com

In this post, we demonstrate how to build a secure, complete LLM fine-tuning workflow that integrates Unity Catalog with Amazon SageMaker AI using Amazon EMR Serverless for preprocessing. The solution

Reuters: Sources: Microsoft is in discussions to acquire LLM developer Inception; SpaceX also courted Inception, which is looking for a price of over $1B — Microsoft (MSFT.O) is shopping for artificia

In this post, we show you how to set up FLOPs tracking during LLM fine-tuning using the open source Fine-Tuning FLOPs Meter toolkit on Amazon SageMaker AI. You learn how to determine your compliance s

Anthropic: Anthropic researchers detail “natural language autoencoders”, which convert LLM activations, the numbers encoding a model's thoughts, into natural language text — When you talk to an AI mod

Tim Harford / Financial Times: A working paper uses an LLM to analyze political discourse on X, finding that anger is the dominant emotion expressed by US users, especially those over 65 — There is a

This Databricks article examines whether Large Language Model agents can effectively optimize SQL join order, a critical query optimization problem in database systems. The analysis likely evaluates L

This article examines the fundamental trade-offs in large language model inference operations, specifically the competing priorities of maximizing throughput, minimizing latency, and reducing costs. I

OpenAI has launched GPT-Rosalind, a biology-tuned large language model . This specialized LLM is designed to enhance performance on biology-specific tasks and applications. The model represents OpenAI

Swathi Moorthy / The Economic Times: Sources say Ola's AI venture Krutrim's smart assistant Kruti is unavailable to users and the development of its Indian multilingual LLM, Krutrim 3, has stalled — W

Speculative decoding is a technique used to accelerate the slow, sequential token generation (decode stage) in LLM inference. This method significantly reduces latency and improves hardware utilizatio

IBM Research has demonstrated that mid-training — a dedicated training phase between initial pre-training and fine-tuning — is essential for improving reasoning capabilities in large language models (

Konstantin Kakaes / Quanta Magazine: How some mathematicians are exploring ways to incorporate LLM models into their research without losing direct experience with mathematical understanding — Those c