Knowledge catalogue
arXiv:2609.04598v1 Announce Type: cross Abstract: We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language
arXiv:2609.04867v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation has advanced rapidly, but its evaluation still underestimates the audio modality. Existing benchmarks either tre
arXiv:2609.05117v1 Announce Type: cross Abstract: Current LLM safety benchmarks largely rely on binary metrics, overlooking how models respond to harmful prompts with varying threat implicitness. We i
arXiv:2601.22931v2 Announce Type: replace Abstract: The prevalence of rapidly evolving slang, neologisms, and highly stylized expressions in informal user-generated text, particularly on Chinese socia
arXiv:2609.01611v1 Announce Type: new Abstract: Frontier large language models can often recognize when they are being evaluated, a capability known as evaluation awareness. If models behave different
Just because someone puts the word AGI in their benchmark doesn't make it even remotely a reflection of what the general public understands by this term. As a layman, I define AGI as a virtual charact
arXiv:2609.01899v1 Announce Type: new Abstract: Multi-camera systems are increasingly practical for robotics, AR/VR, and autonomous driving because complementary views reduce depth ambiguity and prese
arXiv:2609.00242v1 Announce Type: cross Abstract: Long-tail autonomous driving failures are often framed as rare-object recognition errors. We argue that this view is incomplete: the decision-critical
arXiv:2609.00646v1 Announce Type: new Abstract: Commercial short-drama production follows a multi-stage chain: script, storyboard, keyframe imagery, shot-level video, and the finished short drama. Mos
arXiv:2510.15144v4 Announce Type: replace Abstract: Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now appro
arXiv:2609.00192v1 Announce Type: new Abstract: Public trust in Autonomous Vehicles (AVs) may depend not only on technical success but also on the fairness of their decision making. While a recent tre
arXiv:2608.30175v1 Announce Type: new Abstract: Peptide-protein affinity models are often evaluated with a single data split, obscuring whether they interpolate among measurements for observed targets
arXiv:2608.30327v1 Announce Type: new Abstract: Fine-tuning can improve legal question-answering accuracy without improving how models use law supplied in context. We study this distinction in bilingu
arXiv:2608.30678v1 Announce Type: new Abstract: Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across
arXiv:2608.29489v1 Announce Type: cross Abstract: Visual environmental risk recognition plays an important role in secure authentication, where a user's surroundings may reveal sensitive information o
arXiv:2608.29069v1 Announce Type: cross Abstract: Operator-learning surrogates have been benchmarked largely on single-field, single-interface problems, leaving unclear whether architectural choices v
arXiv:2608.29582v1 Announce Type: cross Abstract: Current evaluations of large language models (LLMs) primarily focus on factual knowledge retrieval, overlooking the fundamental challenge of navigatin
arXiv:2608.30688v1 Announce Type: new Abstract: While face recognition systems are widely deployed, ensuring their demographic reliability and robustness under uncontrolled visual conditions remains a
arXiv:2602.08336v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to w
arXiv:2608.29834v1 Announce Type: new Abstract: Collaborative conversations frequently contain references whose targets are indirect rather than named: resolving 'this looks like the fix discussed yes
arXiv:2608.27997v1 Announce Type: new Abstract: Air-ground cross-view referring person detection is a necessary component in the language-to-perception-to-control chain of collective embodied intellig
arXiv:2608.27477v1 Announce Type: cross Abstract: Graphical user interfaces have emerged as an important environment for evaluating autonomous AI agents on multimodal interactive tasks. Existing bench
arXiv:2305.12474v4 Announce Type: replace Abstract: Large Language Models(LLMs) have demonstrated remarkable performance across various natural language processing tasks; however, how to comprehensive
arXiv:2608.26623v1 Announce Type: new Abstract: LLM judges are widely used to evaluate agentic tool-calling systems, yet their reliability on structured, dependency-driven workflows remains largely un
arXiv:2608.27391v1 Announce Type: new Abstract: LLMs are increasingly able to answer complex questions about enterprise-scale document collections. But evaluation is hard: companies don't want to shar
arXiv:2608.26119v1 Announce Type: new Abstract: Whether large language models can be prompted to generate rhetorical fallacies on demand, and whether current safety post-training constrains this behav
Smaller models still struggle with browser tasks that require open-ended research or deep reasoning. But many practical browser-automation workflows are much more procedural: extracting structured dat
arXiv:2608.26535v1 Announce Type: new Abstract: Audio-video generation is rapidly moving from prompt-driven synthesis toward multimodal conditioning, where text, images, audio, and video can jointly s
arXiv:2608.27004v1 Announce Type: new Abstract: Reliable evaluation of vision-language models (VLMs) and medical vision-language models (Medical-VLMs) requires calibrated confidence, particularly unde
arXiv:2608.26292v1 Announce Type: cross Abstract: Every memory-based knowledge editor in the SERAC lineage depends on a scope decision: given a query, does a stored edit apply? We report that current
Google DeepMind: Google launches a pilot of double-blind AI evaluations, keeping external evaluations in a cryptographic “box” to stop benchmark contamination and protect IP — Building trust in propri
arXiv:2601.13207v2 Announce Type: replace Abstract: Geo-localization aims to infer the geographic location where an image was captured using observable visual evidence. Traditional methods achieve imp
arXiv:2608.25854v1 Announce Type: new Abstract: Key Point Analysis (KPA) aims to identify a concise set of key points that summarize a collection of arguments together with their prevalence. We argue
specs hardware: M4 Max 128GB Studio inference engine: oMLX & lllama.cpp insights it still very early, so had to disable oMLX K/V caching, qwen4_exp architectureis not yet supported + the obvious n-gra
arXiv:2608.24314v1 Announce Type: new Abstract: Evaluating conversational voice agents at scale re- quires reliable assessment methods that capture both observ- able interaction quality and the contex
arXiv:2511.00260v2 Announce Type: replace Abstract: Anatomy-aware colonoscopic navigation requires localizing partial endoscopic observations on a stable 3D reference to support coverage assessment, r
arXiv:2608.24086v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as code agents for scientific and engineering analysis, but their ability to analyze raw physical-lay
arXiv:2602.19001v2 Announce Type: replace Abstract: As large language models increasingly power personal assistants, users expect them to reason over multimodal life histories, from recognizing people
arXiv:2608.24486v1 Announce Type: cross Abstract: Purpose: To quantify how evaluation annotations influence measured pulmonary embolism (PE) segmentation performance relative to model training changes
arXiv:2608.24509v1 Announce Type: new Abstract: LLM agents increasingly solve tasks by invoking multiple tools, where parallel execution is essential for low latency but difficult to manage safely. Ex
arXiv:2503.18484v3 Announce Type: replace-cross Abstract: Evaluating the multilingual capabilities of Large Vision-Language Models (LVLMs) remains challenging because most benchmarks rely on non-paral
arXiv:2608.21504v1 Announce Type: new Abstract: The rapid advancement of large language models (LLMs) has led to increasing interest in their application to scientific domains such as chemistry. Howev
arXiv:2608.22842v1 Announce Type: new Abstract: Financial document parsing requires accuracy, structural consistency, and verifiability that current benchmarks often fail to reflect. We present FinixD
arXiv:2608.21928v1 Announce Type: new Abstract: In embodied AI, safety risk can be latent: a benign instruction and a safe scene become hazardous only when composed. Prior work has advanced embodied s
arXiv:2608.22817v1 Announce Type: new Abstract: Industrial technical reports contain high-value knowledge for maintenance, troubleshooting, and product engineering, but their heterogeneous structure (
arXiv:2608.21836v1 Announce Type: new Abstract: Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a
arXiv:2608.22465v1 Announce Type: new Abstract: High-fidelity free-viewpoint video (FVV) and interactive rendering increasingly rely on explicit Gaussian representations, yet practical deployment rema
arXiv:2601.20996v2 Announce Type: replace Abstract: Existing benchmarks for computational materials discovery primarily evaluate static predictive tasks or isolated computational sub-tasks. While valu
arXiv:2608.22039v1 Announce Type: new Abstract: Structure-from-Motion (SfM) is a cornerstone of 3D perception, yet current methods often fail when applied to complex videos involving challenging camer
arXiv:2507.18061v4 Announce Type: replace-cross Abstract: Spoken Language Models (SLMs) are expected to support natural spoken interaction beyond task completion. However, existing SLM benchmarks prim
arXiv:2608.20608v1 Announce Type: new Abstract: Grape leaf disease recognition is important for precision agriculture, enabling early diagnosis, timely intervention, and improved vineyard management.
Graph: A cumulative metric of Speed x Context Size - the higher the dot - the better. Helpful for looking for the most balanced solution. The research relies on DFlash2 Q2 work by AnalogAlok: https://
arXiv:2608.20664v1 Announce Type: new Abstract: DreamBench-SWE is a multi-session benchmark for software-agent memory hygiene in which later software tasks depend on non-inferable evidence from earlie
arXiv:2512.03262v3 Announce Type: replace-cross Abstract: Vibe coding is a new software development paradigm in which human engineers prompt a large language model (LLM) agent to complete complex codi
arXiv:2608.20414v1 Announce Type: new Abstract: Vision-language models are increasingly used for multimodal question answering, yet their ability to reconstruct latent spatial structure from a single
arXiv:2608.20918v1 Announce Type: new Abstract: Organizations maintain task-specific adapters for open-weight language models, and each new base-model release forces a migration decision: retain exist
arXiv:2606.19161v2 Announce Type: replace Abstract: Establishing a universal benchmark for tactile representation learning in robotic manipulation remains challenging due to the diversity of tactile s
tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l
arXiv:2608.18554v1 Announce Type: cross Abstract: Most LLM benchmarks rank models on their ability to automate work tasks. In practice, however, models are often used to assist other (human or LLM) ag
arXiv:2608.18752v1 Announce Type: cross Abstract: Statutory retrieval is necessary for citation-grounded legal question answering, but remains underexplored for Greek. We introduce GreekBarRetrieval,