Reporting benchmark results as a scalar number, e.g. '75% on XYZ' is completely meaningless at this point. You should always report efficien…

Reporting benchmark results as a single scalar percentage is insufficient for meaningful evaluation of model performance. Comprehensive benchmark reporting should include efficiency metrics alongside accuracy scores to provide context about computational costs, latency, and resource requirements. This approach enables fair comparison between models and prevents misleading claims based on accuracy alone.

Source: Francois Chollet (X) | 2026-07-06