Knowledge catalogue
arXiv:2604.04815v2 Announce Type: replace Abstract: The rapid development of Large Language Models (LLMs) has transformed fake news detection and fact-checking tasks from simple classification to comp
arXiv:2604.18418v1 Announce Type: new Abstract: Recent advances in deep research systems enable large language models to retrieve, synthesize, and reason over large-scale external knowledge. In medici
arXiv:2604.18226v1 Announce Type: new Abstract: Automated analysis of customer feedback on social media is hindered by three challenges: the high cost of annotated training data, the scarcity of evalu
arXiv:2604.16909v1 Announce Type: new Abstract: As large language models (LLMs) evolve from conversational assistants into agents capable of handling complex tasks, they are increasingly deployed in h
QIMMA (قِمّة) is a quality-focused leaderboard for evaluating Arabic Large Language Models, developed by TIIUAE. The leaderboard benchmarks Arabic LLMs across various performance metrics and tasks to
arXiv:2604.16396v1 Announce Type: new Abstract: Islamic inheritance law (ilm al-mawar{i}th) presents a challenging domain for evaluating large language models' structured reasoning capabilities, requi
arXiv:2604.17066v1 Announce Type: new Abstract: Coherent systems are representative of many practical applications, ranging from infrastructure networks to supply chains. Probabilistic evaluation of s
arXiv:2604.16593v1 Announce Type: new Abstract: We present SemanticQA, an evaluation suite designed to assess language models (LMs) in semantic phrase processing tasks. The benchmark consolidates exis
arXiv:2503.03480v4 Announce Type: replace Abstract: Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-w
arXiv:2504.18269v2 Announce Type: replace Abstract: When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible. However, th
arXiv:2604.16916v1 Announce Type: new Abstract: Safety alignment in large language models (LLMs) is primarily evaluated under open-ended generation, where models can mitigate risk by refusing to respo
arXiv:2601.04043v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to
arXiv:2604.16767v1 Announce Type: new Abstract: Audio platforms have evolved beyond entertainment. They have become central to public discourse, from podcasts and radio to WhatsApp voice notes and liv
arXiv:2604.15353v1 Announce Type: cross Abstract: This study presents a methodology for identifying the most informative frequencies and channels in electromyography (EMG) data to evaluate muscle reco
arXiv:2604.15647v1 Announce Type: new Abstract: Measuring the quality of public deliberation requires evaluating not only civility or argument structure, but also the informational progress of a conve
arXiv:2604.15866v1 Announce Type: cross Abstract: Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their ge
arXiv:2604.15715v1 Announce Type: cross Abstract: The development of general-purpose agents requires a shift from executing simple instructions to completing complex, real-world productivity workflows
arXiv:2604.15664v1 Announce Type: new Abstract: The rise of autonomous AI agents suggests that dynamic benchmark environments with built-in feedback on scientifically grounded tasks are needed to eval
arXiv:2604.16123v1 Announce Type: new Abstract: Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by sma
arXiv:2604.16272v1 Announce Type: cross Abstract: As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured
Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20 We comprehensively benchmarked Opus 4.7 on document understanding. We evaluated it through ParseBench - our comprehensive
arXiv:2604.14179v1 Announce Type: new Abstract: Rare diseases affect over 300 million people worldwide and are characterized by complex care pathways, limited clinical expertise, and substantial unmet
arXiv:2604.15075v1 Announce Type: cross Abstract: Open-weight Small Language Models(SLMs) can provide faster local inference at lower financial cost, but may not achieve the same performance level as
arXiv:2601.12407v2 Announce Type: replace-cross Abstract: As LLMs rapidly advance and enter real-world use, their privacy implications are increasingly important. We study an authorship de-anonymizati
arXiv:2604.15302v1 Announce Type: cross Abstract: LLM-as-judge frameworks are increasingly used for automatic NLG evaluation, yet their per-instance reliability remains poorly understood. We present a
arXiv:2604.14180v1 Announce Type: new Abstract: We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.56 billion tokens of pure Classical Chinese, with zero Englis
arXiv:2604.15210v1 Announce Type: cross Abstract: Humor is one of the few cognitive tasks where getting the reasoning right matters as much as getting the answer right. While recent work evaluates hum
arXiv:2604.15190v1 Announce Type: cross Abstract: Simulating group-level user behavior enables scalable counterfactual evaluation of merchant strategies without costly online experiments. However, bui
arXiv:2604.02585v2 Announce Type: replace-cross Abstract: LLMs are increasingly used for high-stakes decision-making, yet their sensitivity to spurious contextual information can introduce harmful bia
arXiv:2510.07890v3 Announce Type: replace Abstract: Research on cross-dialectal transfer from a standard to a non-standard dialect variety has typically focused on text data. However, dialects are pri
arXiv:2511.07412v2 Announce Type: replace Abstract: Developing embodied AI for intelligent surgical systems requires safe, controllable environments for continual learning and evaluation. However, saf
arXiv:2604.14548v1 Announce Type: cross Abstract: As speech language models (SLMs) transition from personal devices into shared, multi-user environments, their responses must account for far more than
arXiv:2604.13800v1 Announce Type: new Abstract: Embodied AI research is increasingly moving beyond single-task, single-environment policy learning toward multi-task, multi-scene, and multi-model setti
arXiv:2604.13371v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly described as possessing strong reasoning capabilities, supported by high performance on mathematical, logi
arXiv:2604.13288v1 Announce Type: new Abstract: We present a unified pipeline for synthesizing high-quality Quechua and Spanish speech for the Peruvian Constitution using three state-of-the-art text-t
arXiv:2604.13954v1 Announce Type: new Abstract: Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions.
arXiv:2510.14232v2 Announce Type: replace-cross Abstract: Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models
very excited for this one! a year ago, most of what was being traced to LangSmith was 'LLM apps'. now everything is becoming an agent. with that shift, it's getting harder to know what your software i
This r/StableDiffusion thread discusses community comparisons of video generation models (such as those built on Stable Diffusion or FLUX architectures) evaluated specifically for how well they preser
arXiv:2604.12898v1 Announce Type: new Abstract: Large Language Model-based Hyper Heuristic (LHH) has recently emerged as an efficient way for automatic heuristic design. However, most existing LHHs ju
arXiv:2604.02830v2 Announce Type: replace Abstract: Detecting whether a model's internal knowledge is sufficient to correctly answer a given question is a fundamental challenge in deploying responsibl
arXiv:2604.12081v1 Announce Type: new Abstract: Memory is fundamental to social interaction, enabling humans to recall meaningful past experiences and adapt their behavior accordingly based on the con
Hugging Face co-founder and CEO Clément Delangue advocates for more developers and researchers to focus on building evaluation harnesses and testing frameworks specifically designed for open-source an
arXiv:2512.00412v4 Announce Type: replace-cross Abstract: Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, offering enhanced transparency and logical
arXiv:2604.10853v1 Announce Type: new Abstract: Task-oriented evaluation of knowledge graph (KG) quality increasingly asks whether an ontology-based representation can answer the competency questions
arXiv:2604.10252v1 Announce Type: new Abstract: Reinforcement learning agent-based simulation (RL-ABS) has become an important tool for electricity market mechanism analysis and evaluation. In the mod
arXiv:2604.09819v1 Announce Type: cross Abstract: We introduce ACCIDENT, a benchmark dataset for traffic accident detection in CCTV footage, designed to evaluate models in supervised (IID and OOD) and
arXiv:2601.11044v3 Announce Type: replace Abstract: Large Language Models (LLMs) based autonomous agents demonstrate multifaceted capabilities to contribute substantially to economic production. Howev
arXiv:2604.10547v1 Announce Type: new Abstract: We introduce Agent^2 RL-Bench, a benchmark for evaluating agentic RL post-training -- whether LLM agents can autonomously design, implement, and run com
arXiv:2604.10856v1 Announce Type: cross Abstract: Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in c
arXiv:2509.00891v3 Announce Type: replace Abstract: Real-world adoption of closed-loop insulin delivery systems (CLIDS) in type 1 diabetes remains low, driven not by technical failure, but by diverse
arXiv:2604.11287v1 Announce Type: new Abstract: Background: Large language models (LLMs) have been explored as tools for generating personalized exercise prescriptions, yet the consistency of outputs
arXiv:2604.09682v1 Announce Type: cross Abstract: Autonomous network management in Open Radio Access Networks requires intelligent decision making across conflicting objectives, yet existing LLM based
arXiv:2507.03336v4 Announce Type: replace Abstract: Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the
arXiv:2604.09573v1 Announce Type: cross Abstract: Experimental user studies evaluating the effectiveness of different subtypes of post-hoc explanations for black-box models are largely nonexistent. Th
arXiv:2604.03765v2 Announce Type: replace Abstract: Recent advances in multimodal large language models (MLLMs) have greatly improved image understanding and captioning capabilities. However, existing
arXiv:2602.15481v2 Announce Type: replace Abstract: LLM-as-a-judge has emerged as a cornerstone technique for evaluating large language models by leveraging LLM reasoning to score prompt-response pair
arXiv:2604.10755v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced clinical tasks for common conditions, but their performance on rare diseases remains largely unte
arXiv:2604.10401v1 Announce Type: new Abstract: Inferring nationality from personal names is a critical capability for equity and bias monitoring, personalization, and a valuable tool in biomedical an
arXiv:2504.16355v2 Announce Type: replace-cross Abstract: Perceptual hashing is used to detect whether an input image is similar to a reference image with a variety of security applications. Recently,