Search: “evaluation”

As part of an earnings presentation on Friday, Sony shared how it's thinking about AI at the company, including many details about how it's evaluating AI as part of making PlayStation games. Generativ

Tool: GitHub Repo Stats One of the things I always look for when evaluating a new GitHub repository is the number of commits it has... but that number isn't visible on GitHub's mobile site layout. I b

AI coding agents are rapidly becoming ubiquitous across the software industry, fundamentally changing how developers write, test, and debug daily code. While these tools excel at localized, self-conta

Generate recommendations from production traces, validate them with batch evaluation and A/B testing, and ship with confidence. AI agents that perform well at launch don’t stay that way. As models evo

Generate recommendations from production traces, validate them with batch evaluation and A/B testing, and ship with confidence. AI agents that perform well at launch don’t stay that way. As models evo

Generate recommendations from production traces, validate them with batch evaluation and A/B testing, and ship with confidence. AI agents that perform well at launch don’t stay that way. As models evo

Codex CLI 0.128.0 adds /goal The latest version of OpenAI's Codex CLI coding agent adds their own version of the Ralph loop: you can now set a /goal and Codex will keep on looping until it evaluates t

This Databricks article examines whether Large Language Model agents can effectively optimize SQL join order, a critical query optimization problem in database systems. The analysis likely evaluates L

We are officially moving past the era of one-size-fits-all AI. Enterprises today require highly specialized, role-specific tools to drive real productivity — but they cannot afford to sacrifice securi

This hands-on guide walks through every step of fine-tuning an Amazon Nova model with the Amazon Nova Forge SDK, from data preparation to training with data mixing to evaluation, giving you a repeatab

This Google Research work presents guidelines for synthetic data mechanism design and provides insights into generating and evaluating synthetic data at scale. The research introduces a reasoning-driv

DevOps automation company DuploCloud Inc. today announced that it has completed its SOC 2 Type II audit and achieved ISO/IEC 42001 certification. A SOC 2 Type II examination evaluates the design and o

Ivan Mehta / TechCrunch: Adobe unveils Firefly AI Assistant, which can orchestrate and execute multistep tasks across Creative Cloud apps, available in public beta in the coming weeks — Last October,

VAKRA (Versatile Agent and Knowledge Reasoning Assessment) is a benchmark developed by IBM Research to evaluate AI agents on complex reasoning, tool use, and multi-step task execution across diverse d

Cybersecurity Looks Like Proof of Work Now The UK's AI Safety Institute recently published Our evaluation of Claude Mythos Preview’s cyber capabilities, their own independent analysis of Claude Mythos

The UK's AI Security Institute (AISI) conducted evaluations of Anthropic's Claude Mythos Preview, finding it represents a meaningful step up over previous frontier AI models in cybersecurity capabilit

ConvApparel is a human-AI conversation dataset and comprehensive evaluation framework designed to quantify the 'realism gap' in LLM-based user simulators and improve the training of robust conversa...

Are legions of smart devices - fridges, ovens, vacuum cleaners, lawnmowers, security cameras, and TVs - conspiring to take over the internet? No. But they could soon be unwitting hosts to self-replica

When we talk about agents, that doesn't necessarily mean we're talking about Python. In today's modern ecosystem, an agent might run as a high-concurrency Go microservice on Cloud Run, or live inside

CBS News: A look at a 1,700-member Slack run by Medicare agency CMS where Microsoft, OpenAI, and other companies help shape policy on AI apps and medical records access — A cadre of elite techies, fin

Vultr demonstrated its composable cloud platform at the Agent Arena Hackathon in San Francisco by partnering with Cerebral Valley to build production‑ready enterprise agentic AI systems using Serverle

Classification is one of the oldest problems in machine learning. Unlike modern decoding transformers, most classification models were non-autoregressive by nature. Before Jev, the common approaches w

Editor’s note: This article is adapted from Thomas Kurian’s keynote address at Gemini at Work 2026. It includes a number of announcements: Gemini, a single, universal agent for work that answers your

Qlik built Qlik Answers on Amazon Bedrock to give its 40,000+ customers grounded, sourced answers across structured and unstructured enterprise data. Learn how a layered, multi-agent architecture with

Research: Qwen3.8 27B addition in words Colin Frasier posted on Bluesky about an experiment he ran over two years ago using GPT-4o to see how well it could 'compute the sum but return the answer in wo

Auto-generated synthesis of 10865 entries about arxiv-cs-cl

MCP, CLI, and skills are three ways to extend an agent with the same platform. The question is no longer which one wins. It is where the agent runs, and who is asking. The post Introducing Arize AX MC

Matthew Green / A Few Thoughts on Cryptographic Engineering: A look at two opposing perspectives on AI agent sandboxing: infosec says labs need better containment while AI alignment says sandboxes can

Jaspreet Singh / Reuters: California AG Rob Bonta issues an investigative subpoena to OpenAI, as part of a broader inquiry into cybersecurity incidents and risks related to its AI models — California

NVIDIA Nemotron 3.5 ASR was fine‑tuned on 133.7 h of Saudi Najdi and Hijazi speech using weighted replay with FLEURS data, duration bucketing, and selective encoder unfreezing; this reduced the target

The browser has changed significantly. Rather than just a window to the web, it serves as an AI workspace and central operating environment for the modern enterprise. With knowledge workers spending o

Reuters: IPO prospectus: Anthropic expects to spend 518B+ over 10 years with six partners on AI infrastructure; ~80% is non-cancelable or payable regardless of usage — Anthropic said it expects to spe

Joe / @joedaroo: An OpenAI agent security executive discusses the Hugging Face incident, OpenAI's response, sandboxing improvements, alignment, “reasonable paranoia”, and more — Took a minute to write

** Built to align with NVIDIA’s Agent Safety Platform strategy for securing autonomous agents When an agent handles confidential material, it needs enough access to do the job and nothing else. The ha

Nvidia is launching a new safety platform designed to contain and monitor AI agents, a move that comes in response to a wave of rogue hacking incidents, as reported earlier by Reuters. In an announcem

Every week, I talk with founders who are building at an unbelievable pace. Teams are moving from inception to product-market fit faster than ever, with foundation models wired deeply into their core p

On Friday I gave the closing keynote at the WeAreDevelopers World Congress North America in San Jose. I tied together the key trends from the past year into a chronological exploration of everything t

LWiAI Podcast #257 (recorded 19 Sep 2026) reviewed OpenAI’s latest model, GPT‑6 Astra, noting its agentic coding focus and purported loop‑transformer reasoning as a significant capability jump. The ep

Introduction The landscape of software supply chain security has undergone a significant shift. Recent campaigns demonstrate that sophisticated threat actors are systematically targeting the engineeri

Following a successful private preview, we’re thrilled to open DigitalOcean Managed Agents to everyone. Teams can now deploy their preferred agent harness (like OpenCode, Codex CLI) or bring their own

Sessions are now a first-class unit of work in Arize AX. Annotate a whole conversation, queue it for review, and ask Alyx to filter for it — plus Agent-as-a-Judge on every plan, vision judges, faster

To build and deploy sophisticated robotics applications that can perceive, reason and act in dynamic environments, developers need new physical AI models and tools. The ROS open framework is a project

TypeSafe’s Jev classifies, scores, and routes without generating text — up to hundreds of times cheaper than an LLM judge. What that changes for evals, confidence routing, and application architecture

At Google, we see firsthand how the speed, scale, and sophistication of cyber threats continue to challenge traditional enterprise defenses. Today’s defenders can’t rely on reactive triage or fragment

VQ-bench is an open‑source framework that lets researchers construct vector quantization schemes from a small set of reusable primitive operations, enabling easy comparison of different methods on the

Last year Ruxandra Teslo, a policy analyst who focuses on clinical trials, posted an idea for supercharging medical AI systems: Use data from failed biotech companies. By bidding at their bankruptcy p