Search: “evaluation”

arXiv:2607.18269v2 Announce Type: replace Abstract: The wisdom of crowds -- the finding that aggregating judgments across individuals often outperforms the best individual -- has been extensively stud

local-aiarxiv-cs-ai

24 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Zagreus-0.4B-por a small open source language model for Portuguese

DGX agent

mii-llm, an open source AI lab, released Zagreus-0.4B-por, a compact bilingual Portuguese–English language model pretrained entirely from scratch. The model has approximately 400 million parameters an

model-releasesr-localllama

24 Jul 2026

Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents

arXiv:2607.14573v3 Announce Type: replace Abstract: Payment integration is a demanding repository-level software task: agents must select a suitable product, implement coordinated client-server flows,

Tutorials

Auditing Retrieval-Augmented LLM Hypotheses for Longitudinal Cell Painting Morphology

arXiv:2607.19415v1 Announce Type: cross Abstract: High-content morphological profiling (Cell Painting) yields sensitive, high-dimensional signatures of cellular state, but translating longitudinal mor

tutorialsarxiv-cs-ai

Domain-Adapted Power Curve for Cross-Farm Applications

arXiv:2607.19744v1 Announce Type: cross Abstract: The wind energy industry relies on accurate power curve models to make power forecast, evaluate turbine performance, quantify upgrade, or support site

researcharxiv-cs-lg

GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks

arXiv:2512.24592v3 Announce Type: replace Abstract: Systematic failures of vision models on semantically coherent subsets, known as error slices, reveal limitations in robustness and evaluation. Exist

Global Building Area Estimation Products: How Accurate Are They?

arXiv:2607.19766v1 Announce Type: new Abstract: Geo-spatial rasters of building footprint area are useful for a variety of tasks, such as monitoring urbanization, improving energy efficiency, and trac

safetyarxiv-cs-cv

Hybrid LLM-Guided Search for Quantum Reservoir Architecture Design

arXiv:2607.19506v1 Announce Type: cross Abstract: Quantum reservoir computing (QRC) uses fixed quantum dynamics as a high-dimensional temporal feature map and trains only a lightweight classical reado

LKValues: Aligning Large Language Models with Sri Lankan Societal Values

model-releasesarxiv-cs-cl

arXiv:2607.20410v1 Announce Type: new Abstract: Value alignment of Large Language Models (LLMs) has been shown to be culturally biased toward Western norms. This results in the mishandling of local va

Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches

arXiv:2607.18882v1 Announce Type: new Abstract: Validating Explainable Artificial Intelligence (XAI) methods in medical imaging requires ground-truth data with known locations of informative features.

MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction

arXiv:2607.19910v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs

Applications

Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering

arXiv:2607.19856v1 Announce Type: cross Abstract: FinMMEval 2026 Task 1 evaluates multilingual financial multiple-choice question answering in English, Chinese, Arabic, and Hindi. The task tests wheth

applicationsarxiv-cs-ai

Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering

arXiv:2607.19867v1 Announce Type: cross Abstract: FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence. Each final-test item pairs an English question w

[Paper] SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

model-releasesr-localllama

Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlappe

Profile-Graph Memory for LLM Agents: Implicit Cross-Entity Traversal through Narrative Profiles

arXiv:2607.19359v1 Announce Type: new Abstract: Long-term memory is essential for LLM agents that interact across sessions, yet current memory benchmarks primarily evaluate single-hop recall, leaving

Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models

arXiv:2512.11899v2 Announce Type: replace Abstract: Large vision-language models (LVLMs) are vulnerable to typographic attacks, where misleading text inserted into an image can override visual underst

Local Ai

Real-Time EEG Cap Electrode Detection for Guided Point-of-Care Placement

arXiv:2607.20142v1 Announce Type: new Abstract: We present a two-stage vision system that detects EEG cap electrodes in a live webcam stream and validates their anatomical placement in real time. A si

local-aiarxiv-cs-cv

RS-RIE-Bench: Benchmarking Reasoning-Guided Remote Sensing Image Editing

arXiv:2607.20197v1 Announce Type: new Abstract: Remote sensing image editing aims to modify remote sensing images according to natural language instructions while preserving geographic rules and senso

The Two-Process Theory of Machine Self-Report

arXiv:2607.20082v1 Announce Type: new Abstract: Language models are increasingly asked to self-report, informing safety evaluations, public understanding, and model-welfare debates. Yet their reports

safetyarxiv-cs-cl

Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review

arXiv:2507.10142v2 Announce Type: replace Abstract: Multi-Agent Reinforcement Learning (MARL) has achieved strong performance in simulated benchmarks, yet real deployments often violate the assumption

safetyarxiv-cs-ai

Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

arXiv:2607.19442v1 Announce Type: cross Abstract: Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes. In a controlled nonce-fact testbed with a matched retrainin

Available today in the langchain-ai/langchain-skills repo: https://github.com/langchain-ai/langchain-skills 💻 Install in Codex or Claude Co…

model-releasesharrison-chase--x

Available today in the langchain-ai/langchain-skills repo: https://github.com/langchain-ai/langchain-skills 💻 Install in Codex or Claude Code, open the repo w/ the agent you want to evaluate, start wi

22 Jul 2026

Holy shit wow

model-releasesconnor-leahy--x

Holy shit wow We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Shari

22 Jul 2026

NuExtract3 is now available on Ollama: 4B VLM for document-to-Markdown and structured JSON extraction

Disclosure: I work at NuMind, the team that trained NuExtract3. NuExtract3 is an Apache-2.0, open-weight 4B VLM based on Qwen3.5-4B. It is specialized for document understanding rather than general ch

model-releasesr-ollama

22 Jul 2026

We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face p…

model-releasesclem-delangue--x

We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Sharing preliminary

21 Jul 2026

MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

arXiv:2607.09142v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in online medical consultation, yet existing benchmarks remain poorly aligned with real clini

Local Ai

Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning

arXiv:2607.13826v1 Announce Type: cross Abstract: Accurate determination of pancreatic ductal adenocarcinoma (PDAC) resectability relies on evaluating how the tumor interacts with major peripancreatic

local-aiarxiv-cs-ai

Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows

arXiv:2607.13078v1 Announce Type: cross Abstract: LLMs are now proposed for fraud detection, scam investigation, content moderation, and other trust-and-safety workflows. Much of the public literature

safetyarxiv-cs-ai

Privacy Preserving Recommender Systems Balancing Personalization with Privacy

arXiv:2607.13328v1 Announce Type: cross Abstract: Personalized recommendation systems are central to modern e-commerce and retail platforms, but they typically rely on centralized storage of detailed

safetyarxiv-cs-ai

Agents

Self-Improvements in Modern Agentic Systems: A Survey

arXiv:2607.13104v1 Announce Type: new Abstract: Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, fro

agentsarxiv-cs-ai

Hardware

Text2Sign: A Single-GPU Diffusion Baseline for Text-to-Sign Language Video Generation

arXiv:2607.13164v1 Announce Type: cross Abstract: Sign language is a primary communication channel for millions of Deaf and hard-of-hearing people, yet text-to-signer video generation remains costly b

hardwarearxiv-cs-cv

VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling

arXiv:2607.13929v1 Announce Type: new Abstract: Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic input

researcharxiv-cs-lg

Agents

AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP

arXiv:2607.11098v2 Announce Type: replace-cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work. When a tool times out, returns a week-stale value, or has its description

agentsarxiv-cs-ai

Connected by Construction: Learning Tractable Near-Tour Marginals for Traveling Salesman Problems

arXiv:2607.12127v1 Announce Type: new Abstract: Learning-based methods for the traveling salesman problem (TSP) are often evaluated through the tours produced after decoding or search, but the learned

CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?

arXiv:2511.09483v3 Announce Type: replace Abstract: While multimodal large language models can describe visual content, their ability to generate executable procedures remains underexplored. CrochetBe

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

arXiv:2602.05513v3 Announce Type: replace-cross Abstract: Bimanual dexterous manipulation relies on integrating multimodal inputs to perform complex real-world tasks. To address the challenges of effe

Egocentric Bias in Vision-Language Models

arXiv:2602.15892v2 Announce Type: replace-cross Abstract: Visual perspective taking--inferring how the world appears from another's viewpoint--is foundational to social cognition. We introduce FlipSet

First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations

arXiv:2512.01241v4 Announce Type: replace-cross Abstract: Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

arXiv:2607.12894v1 Announce Type: new Abstract: Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, a

Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts

arXiv:2607.11656v2 Announce Type: replace-cross Abstract: Accurate diagnostic classification and disease-severity prediction for Alzheimer's disease are hampered by the incompleteness and heterogeneit

safetyarxiv-cs-lg

LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes

arXiv:2607.12310v1 Announce Type: cross Abstract: While modern question answering (QA) systems excel on clean, schema-aligned corpora, real-world knowledge is rarely so neatly packaged. Answering ques

RepTran: Search-Based Repair of Transformer Models

arXiv:2607.11193v2 Announce Type: replace-cross Abstract: To ensure the overall quality of AI-enabled software, not only traditional software components but also AI components need to be tested and re

Robustness of Deep Learning Models for PV Power Forecasting under NWP Forecast Errors: A Spatiotemporal and Physically Interpretable Analysis

arXiv:2607.12954v1 Announce Type: cross Abstract: Engineering use of AI forecasting models requires not only high nominal accuracy but also predictable behavior under uncertain inputs. In photovoltaic

researcharxiv-cs-lg

The Computational Basis of Confidence in Large Language Models

arXiv:2607.12447v1 Announce Type: cross Abstract: Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existin

Towards Self-Evolving Agents: A Human-Inspired Adaptive Exploration-Exploitation Framework for Genetic Network Programming

arXiv:2607.11913v1 Announce Type: cross Abstract: Recent advancements in agentic AI have increasingly moved toward graph-based methods, driven by the demand for explainable, human-centered, and non-li

UniVR: Thinking in Visual Space for Unified Visual Reasoning

arXiv:2607.12800v1 Announce Type: new Abstract: Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation in

VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression

arXiv:2607.12756v1 Announce Type: new Abstract: Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference latency and memory overhead. This has motivated

We trained and released DSpark speculators for Kimi-K2.6 and Kimi-K2.7-Code on @huggingface, with native serving support in @vllm_project. A…