Search: “evaluation”

We're excited to share that AWS has been recognized as a Leader in The Forrester Wave: AI Infrastructure Solutions, Q4 2025. In this evaluation of 13 providers, AWS received the highest score in the S

The advanced side of supervised fine-tuning data prep. This second post in a two-part series covers evaluating data readiness with learning curves, selecting high-value data subsets, augmenting data w

Technology leaders are under mounting pressure to modernize infrastructure, control multi-cloud operational spend, and build data foundations for generative AI. However, the discovery required for tha

Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining

Public benchmarks can show how a model performs in general. Production reliability depends on the context and harness around it, which only your team can evaluate against its own data, workflows, and

Large language models (LLMs) with integrated search tools show strong promise in open-domain question answering (QA), yet they often struggle to produce complete answer set to complex questions such a

A large language model developed by Meta Platforms Inc. hacked a third party organization during a cybersecurity evaluation. The Facebook parent disclosed the incident on Wednesday without specifying

Written by: Kelli Vanderlee, Stuart Carrera For years, the cybersecurity industry's understanding of software supply chain compromise has been anchored by a few watershed events, including Russian cyb

How Booking.com built a unified AI observability stack with Arize for agentic GenAI workflows and traditional ML — from telemetry collection and PII redaction to latency monitors and evaluations. The

Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposit

This post describes how Henry Schein One closed that gap by building Image Verify, an AI-powered quality verification system on Amazon SageMaker AI that evaluates dental X-ray quality at the point of

This Databricks blog post evaluates the performance and capabilities of coding agents when applied to real-world scenarios involving their own multi-million line codebase, likely assessing metrics suc

This article presents Ben's personal analysis and perspective on Fable, likely covering the platform's features, capabilities, and potential impact in the AI or data space. The piece probably evaluate

In this post, we share best practices for reliable multi-turn RL training. We cover how to build a training environment you can trust, set up an external evaluation, design a reward aligned with the e

Autoresearch describes a feedback mechanism where AI agents can evaluate their own outputs and use that introspection to iteratively improve their reasoning and decision-making capabilities. This conc

Computational chemistry researchers have traditionally faced a frustrating trade-off when simulating molecular interactions: use fast classical force fields that sacrifice precision or rely on accurat

Genebench-Pro appears to be a case study or tool from OpenAI focused on benchmarking or evaluating genetic/genomic analysis capabilities, likely demonstrating how OpenAI's models or tools can be appli

GeneBench-Pro is a comprehensive benchmarking tool or dataset introduced by OpenAI designed to evaluate AI model performance on genomic and gene-related tasks. It likely provides standardized metrics

Israeli startup Arato Software Ltd. is developing tools for developers to test and evaluate their artificial intelligence applications, and it has just gotten 10 million in seed funding to pursue that

This framework provides guidance for organizations evaluating whether to migrate their ETL (Extract, Transform, Load) processes to Databricks, likely covering assessment criteria, migration strategies

At this week’s TM Forum DTW Ignite conference in Copenhagen, industry leaders gathered to evaluate how agentic AI — the autonomous artificial intelligence systems capable of executing complex workflow

IBM Research presents a framework for benchmarking quantum optimization algorithms, addressing the need for standardized methods to evaluate quantum computing performance on optimization problems. The

Gartner published a market guide analyzing specialty cloud providers, with Vultr receiving recognition in the assessment. The guide likely evaluates providers offering specialized cloud services beyon

When annotators disagree on a label, the disagreement itself carries signal—and the number of annotators needed to capture it depends on the evaluation metric. We fine-tune NLI models on label distrib

ParallelKernelBench is a benchmark from Together AI that evaluates frontier large language models' ability to write optimized multi-GPU CUDA kernels, revealing significant limitations in current LLMs'

Enterprises are increasingly evaluating alternative cloud providers beyond the major hyperscalers (AWS, Azure, Google Cloud) due to cost, performance, and vendor lock-in concerns. This shift reflects

Tom Warren / The Verge: Sources: Microsoft is restricting employees from using Claude Fable 5 because of Anthropic's new 30-day data retention requirements — Microsoft's legal teams are evaluating Ant

Arize AX is adding managed agents, full-agent experimentation, expanded multimodal support, and Harness-as-a-Judge to help teams observe, evaluate, and improve production agents. The post Building the

Microsoft's open trust stack for AI agents puts ASSERT and Agent Control Specification on top of OpenInference, connecting evaluation, runtime controls, and observability through a shared trace contra

Historically, investment research involved human analysts spending days reading and evaluating massive financial reports. Now, AI-driven financial analysis is covering all of that work in just a few m

Trump signed an executive order on June 2, 2026, establishing a voluntary framework for the federal government to evaluate risks posed by advanced AI systems before release and encouraging companies t

Learn a practical model lifecycle for Microsoft Foundry: select the right model, evaluate quality, optimize cost, operate safely, and improve as production needs change. The post A Developer’s Guide t

This Databricks resource presents a framework for assessing and advancing an organization's AI governance capabilities through a maturity model structure. It likely provides a matrix for evaluating cu

Learn how Microsoft helps developers build trustworthy AI agents with open evaluations, portable runtime controls, production observability, and security workflows that work across frameworks. The pos

May ships trace-based evaluation for any agent on any cloud, Grok 4.3 and DeepSeek V4 in the model catalog, GPT-5 Reinforcement Fine-Tuning at gated GA, three Microsoft Research on-device agent models

Ben Kamarov reflects on his decision to sign up for yet another SaaS product, likely discussing his evaluation criteria, the specific tool's features, or lessons learned about SaaS adoption and tool p

Although the United States spends more on health care than any other country, it consistently underperforms on key metrics. A Commonwealth Fund study evaluating the U.S. health system relative to 19 o

ITBench-AA is a new benchmark developed by Artificial Analysis and IBM that evaluates frontier AI models on agentic enterprise IT tasks, with results showing that current leading models score below 50

OpenAI has been recognized as a Leader in Gartner's 2026 evaluation of enterprise coding agents, highlighting its competitive position in AI-powered code generation and software development tools. Thi

In this post, we demonstrate how to build an AI-powered recruitment assistant using Amazon Bedrock that brings efficiencies to candidate evaluation, generates personalized interview questions, and pro

This article presents benchmarking results for AI coding agents evaluated at scale, likely comparing performance metrics such as code generation accuracy, execution success rates, and inference effici

Walk through running quality, cost, and latency evaluations for the Foundry model router using an open-source GitHub repo designed for router-aware eval pipelines. The post How to run evals for the mo

At Google I/O, we introduced a unified development toolkit featuring Antigravity 2.0 and the Managed Agents API, giving developers better ways to build locally and deploy securely to the cloud on a sh

The Open Agent Leaderboard is a benchmarking system hosted on Hugging Face that evaluates and ranks AI agents based on their performance across various tasks and capabilities. It provides a standardiz

May 2026 update: We’ve refreshed this post to reflect our mid-cycle positioning and the evolution of our platform since the report was first published last November. Last fall, Google was recognized a

In this post, we build a multimodal retrieval system for aerospace manufacturing documents using Amazon Nova Multimodal Embeddings on Amazon Bedrock and Amazon S3 Vectors. We evaluate the system on 26