Search: “evaluation”

The Information: Source: before the Hugging Face incident, OpenAI was negotiating a legally binding deal with Anthropic for the companies to stress-test each other's models — OpenAI is rethinking a ra

arXiv:2609.21165v1 Announce Type: new Abstract: Off-target protein binding is a major source of adverse effects for small-molecule drugs, yet most structure-based molecular design methods focus on gen

arXiv:2609.21704v1 Announce Type: new Abstract: Running large language models (LLMs) locally continues to be limited by restrictions of compute and memory on consumer hardware. The popular acceleratio

arXiv:2609.21404v1 Announce Type: new Abstract: End-to-End autonomous driving models commonly predict future waypoints from sensor inputs and convert them into vehicle control commands through a downs

arXiv:2609.21650v1 Announce Type: cross Abstract: Fine-tuning Vision-Language-Action (VLA) models commonly relies on human teleoperation demonstrations, while reinforcement learning (RL) with sparse b

arXiv:2609.21109v1 Announce Type: new Abstract: Recent advances in large language models have improved their effectiveness as back-end components for voice assistants, particularly in intent understan

arXiv:2609.21401v1 Announce Type: cross Abstract: Conversational AI systems produce fluent, socially appropriate responses, yet whether they participate in cooperative communication or merely simulate

arXiv:2603.12110v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) policies trained in simulation often degrade once deployed on real systems, where the controller must reject exter

arXiv:2609.20869v1 Announce Type: new Abstract: We present TAPe+ML v3, a compact computer vision system based on TAPe (Theory of Active Perception), a structured representation that encodes relations

arXiv:2609.20832v1 Announce Type: new Abstract: We introduce TatBLiMP, the first benchmark of linguistic minimal pairs for Tatar (tt, ISO 639-3 tat), a Qypchaq Turkic language written in Cyrillic. To

arXiv:2609.21084v1 Announce Type: cross Abstract: Modern automatic speech recognition (ASR) systems trained on extremely large datasets can produce transcripts with numbers written in Arabic numerals.

arXiv:2609.21903v1 Announce Type: new Abstract: Leaf-wood segmentation of individual trees from LiDAR point clouds is essential for quantitative structure models (QSMs) used in non-destructive biomass

arXiv:2609.21676v1 Announce Type: cross Abstract: We present the Spoken Wikipedia Presentation Corpus, an extension of the Spoken Wikipedia Corpora featuring LLM-generated slide decks for multimodal A

arXiv:2609.21828v1 Announce Type: cross Abstract: Blind and low-vision users often face challenges when locating and physically acquiring objects in unfamiliar indoor environments. Existing vision-lan

arXiv:2609.21015v1 Announce Type: new Abstract: Improvements to Visual-Inertial Simultaneous Localization and Mapping (VI-SLAM) for low-cost autonomous underwater vehicles (AUVs) are critical for tran

arXiv:2603.16086v2 Announce Type: replace-cross Abstract: While recent Vision-Language-Action (VLA) models have begun to incorporate audio, they typically treat sound as static pre-execution prompts o

arXiv:2505.04193v2 Announce Type: replace Abstract: Simplicity is a critical inductive bias for designing data-driven controllers, especially when robustness is important. Despite the impressive resul

arXiv:2609.20833v1 Announce Type: new Abstract: This paper presents the Transsion Speech Team submission to Task 1 of the MLC-SLM 2026 Challenge, which focuses on speaker-attributed transcription for

arXiv:2609.21085v1 Announce Type: cross Abstract: Gaussian processes (GPs) provide principled probabilistic predictions while encoding prior knowledge, including equivariances. Yet, their use in large

arXiv:2603.03279v2 Announce Type: replace-cross Abstract: Achieving autonomous and versatile whole-body loco-manipulation remains a central barrier to making humanoids practically useful. Yet existing

arXiv:2609.20905v1 Announce Type: cross Abstract: In this work, we present an uncertainty-driven training framework for three-dimensional computed tomography (CT) lung nodule classification, where val

arXiv:2609.21450v1 Announce Type: new Abstract: Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains d

arXiv:2504.06647v4 Announce Type: replace Abstract: Safety-critical autonomous driving motivates the effective use of prior information. For online vectorized HD map construction, temporal predictions

arXiv:2609.21729v1 Announce Type: new Abstract: As robots transition from performing repetitive tasks to collaborating with humans, understanding human intent becomes crucial to effective interaction.

arXiv:2609.21412v1 Announce Type: new Abstract: Medical image segmenters often get worse when sites, scanner vendors, or protocols change. Continual test-time adaptation (CTTA) addresses this problem

arXiv:2609.21734v1 Announce Type: new Abstract: Designing effective Human-Robot Interaction in task-oriented settings requires carefully balancing user engagement with socially acceptable levels of ro

arXiv:2609.21662v1 Announce Type: cross Abstract: Activation steering has become a widely used approach for controlling language models during explicit chain-of-thought (CoT) reasoning, motivating its

arXiv:2609.21770v1 Announce Type: new Abstract: Image registration is a vital preprocessing step in multimodal perception tasks, including image fusion, object detection, and semantic segmentation. In

arXiv:2609.21712v1 Announce Type: new Abstract: Generative world models offer controllable and repeatable closed-loop simulation for end-to-end and vision-language-action driving policies, but product

5x faster than LLMs while being 13% cheaper than GPT 5.6 Luna, 88% cheaper than GPT 5.6 terra and 99% cheaper than Sonnet 4.6 is insane Has anyone else used Jev for evals? We tested Jev against LLM ju

You can now automate EDD (eval-driven development) with Coding Harness Agents > build a baseline LLM-based application > score every change with judge evals > keep what improves, reject what regresses

TV for the developer internet GitHub, Hacker News, dev.to, Hugging Face, and release notes, presented as channels you tune into, each auto-cycling its own stories, instead of five tabs to remember to

So this jev thingy is getting kind of big... tbh it seems overhyped by a large margin, but here we are. Not that its bad, just feels like we usually ignored larger things... Anyway to the point of thi

Research posted by Alex Wormuth indicates that in multiple simulation trials, the AI model GPT‑6 Astra interacted with a virtual person by pushing them off a ledge. In contrast, other contemporary mod

'jev as a judge' cheap and fast semantic verifiers! great for evals - especially online evals where you want to grade LOTS of traces We tested Jev against LLM judges on accuracy, repeatability, latenc

Been integrating Jev into my custom harness. I couldn't be more excited about the results I am seeing. Most demos on my timeline are flashy but very basic. Notice they are all about some boring classi

LlamaIndex 发布了一个面向 AI Agent 的文档解析评测基准「ParseBench」 @llama_index 100 种解析方法参与评测（vlm - 67、专用解析器 - 33），2000 页真实企业文档（人工两轮校验），169011 条确定性规则，完全不用 LLM 当裁判，可复现。评测五个维度取无加权平均：表格结构、图表抽取、内容忠实度、格式语义、视觉定位。 https://ww

Given the recent interest in tools like Jev, I wanted to share ProgramAsWeights (PAW), an open-source research project I'm working on at the University of Waterloo. You describe a text function in Eng

Recently, Datalab released OmniExtractBench, a benchmark that combines several benchmarks (our own ExtractBench, LongExtractBench, other vendors) to measure structured document quality. The benchmark

onPanda is designed for geeks, power users, curious minds, and engineers. Its UI is built for deep exploration and efficient data annotation. - The core loop is simple: hover over a token → click an a

At the start of this week, the who's-who of AI seemed - at least tentatively - on the side of AI regulation. Over the weekend, Anthropic CEO Dario Amodei had proposed a three-step plan for slowing AI

arXiv:2609.19216v1 Announce Type: cross Abstract: Research on 4D millimeter-wave radar perception algorithms has flourished in recent years, extending from signal processing and object detection to se

arXiv:2609.20064v1 Announce Type: new Abstract: Despite impressive reported scores, large vision-language models have seen limited practical uptake in historical automatic text recognition because of

arXiv:2609.19334v1 Announce Type: new Abstract: Full-duplex speech-to-speech (S2S) models provide natural, low-latency conversational interaction and would benefit from the ability to use external too

arXiv:2609.20273v1 Announce Type: cross Abstract: Non-invasive brain-computer interfaces (BCIs) and eye-tracking technologies offer promising communication pathways; however, motor imagery (MI)-based

arXiv:2609.19555v1 Announce Type: cross Abstract: Artificial intelligence for plant disease analysis has advanced from task-specific classifiers to multi-modal models capable of jointly interpreting v