Knowledge catalogue
arXiv:2604.14388v1 Announce Type: new Abstract: Humans routinely infer taste, smell, texture, and even sound from food images a phenomenon well studied in cognitive science. However, prior vision lang
arXiv:2604.13201v1 Announce Type: new Abstract: Large language models are emerging as scientific assistants, but evaluating their ability to reason from empirical data remains challenging. Benchmarks
arXiv:2604.12379v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly rely on explicit reasoning to solve coding tasks, yet evaluating the quality of this reasoning remains chall
arXiv:2604.12290v1 Announce Type: new Abstract: Current LLM agent benchmarks, which predominantly focus on binary pass/fail tasks such as code generation or search-based question answering, often negl
arXiv:2604.12843v1 Announce Type: new Abstract: The rapid release of both language models and benchmarks makes it increasingly costly to evaluate every model on every dataset. In practice, models are
arXiv:2604.09819v1 Announce Type: cross Abstract: We introduce ACCIDENT, a benchmark dataset for traffic accident detection in CCTV footage, designed to evaluate models in supervised (IID and OOD) and
arXiv:2604.10454v1 Announce Type: new Abstract: Affective Image Manipulation (AIM) aims to evoke specific emotions through targeted editing. Current image editing benchmarks primarily focus on object-
arXiv:2602.21950v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have shown great potential in medical applications, yet existing benchmarks inadequately capture real-world
ParseBench is the most comprehensive OCR benchmark for real-world enterprise documents: financial filings, contracts, insurance documents, and more. We evaluate across 5 dimensions that are present am
arXiv:2512.19691v3 Announce Type: replace Abstract: Reference labels for machine-learning benchmarks are increasingly synthesized with LLM assistance, but their reliability remains underexamined. We a
arXiv:2604.10291v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown promising performance in time series modeling tasks, but do they truly understand time series data? While multip
arXiv:2602.14200v4 Announce Type: replace Abstract: Time Series Language Models (TSLMs) are emerging as unified models for reasoning over continuous signals in natural language. However, long-context
arXiv:2604.10542v1 Announce Type: cross Abstract: Video-to-Audio (V2A) generation is essential for immersive multimedia experiences, yet its evaluation remains underexplored. Existing benchmarks typic
This r/MachineLearning post discusses the landscape of frameworks and tools used to support benchmarking of LLMs and agentic AI systems, covering how to systematically evaluate model capabilities beyo
arXiv:2604.07017v1 Announce Type: new Abstract: AI assistants that interact with users over time need to interpret the user's current emotional state in order to respond appropriately and personally.
arXiv:2604.08540v1 Announce Type: cross Abstract: Text-to-Audio-Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing benchma
arXiv:2604.08287v1 Announce Type: new Abstract: Discovering camouflaged objects is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings.
arXiv:2511.19474v4 Announce Type: replace-cross Abstract: Automatically detecting abnormal events in videos is crucial for modern autonomous systems, yet existing Video Anomaly Detection (VAD) benchma
The new model from Meta is already looking like a disappointment: overoptimized for public benchmark numbers at the detriment of everything else. Knowing how to evaluate models in a way that correlate
A lot of you are asking about the :cloud tag. Here's the deal: GLM-5.1 is a 744B parameter beast. To hit that 54.9 benchmark score without turning my M4 Mac Mini into a space heater, I run the cloud e
This naming scheme is bonkers (minor model numbers and date tags) and the benchmark step is big. Could prove stronger than GLM 5.3 with vision built in. The King of open weights is back? 🚀 DeepSeek-V4
arXiv:2610.20785v1 Announce Type: new Abstract: Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental stat
arXiv:2610.21038v1 Announce Type: cross Abstract: Machine learning models for tabular prediction typically assume that training data are representative of the population on which they will be deployed
arXiv:2610.21082v1 Announce Type: new Abstract: Large language model (LLM) agents can keep scientific experiments running for months with little human supervision, but reliable autonomous research als
arXiv:2610.21902v1 Announce Type: new Abstract: Language priors in MLLMs can override visual evidence, impairing visual perception. Existing benchmarks assume shared priors across models and rely sole
arXiv:2610.17027v1 Announce Type: new Abstract: Detecting credit card fraud is notoriously difficult because fraudulent charges make up less than 0.2% of total transactions. Quantum Machine Learning (
arXiv:2602.08519v2 Announce Type: replace Abstract: Attributed Graph Clustering (AGC) is a fundamental unsupervised task that integrates structural topology and node attributes to uncover latent patte
arXiv:2506.23101v2 Announce Type: replace-cross Abstract: Ensuring the trustworthiness of Multimodal Large Language Models (MLLMs) requires rigorous evaluation of gender bias, particularly in socially
arXiv:2610.20370v1 Announce Type: new Abstract: Hate speech has become increasingly prevalent on social media, raising concerns about its detection, interpretation, and moderation. Persian, a relative
arXiv:2610.15994v1 Announce Type: new Abstract: Representing individual-level behavioral heterogeneity is challenging in epidemiological modeling, particularly when empirical behavioral data are spars
arXiv:2610.20027v1 Announce Type: cross Abstract: Linters are widely used in code maintenance and real-time IDE editing to surface lint alerts related to performance, style, error-prone, and other pat
arXiv:2610.15714v1 Announce Type: new Abstract: Healthcare agents must transform longitudinal clinical evidence into reliable actions through heterogeneous tools and stateful workflows, conditions und
arXiv:2610.19190v1 Announce Type: new Abstract: Many sentiment datasets use one annotator for each item, so the test labels show the conventions of that person. We build a Nepali benchmark from two an
arXiv:2610.17288v1 Announce Type: new Abstract: Building physical simulations from video requires coding agents to translate observations into executable models and refine them through tool use. We in
arXiv:2412.12877v3 Announce Type: replace Abstract: Recent Artificial Intelligence (AI)-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the
arXiv:2610.19910v1 Announce Type: new Abstract: Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rul
arXiv:2610.15666v1 Announce Type: new Abstract: Converting natural-language instructions into executable action plans conditioned on a scene image is a core capability for embodied human-robot interac
arXiv:2610.14647v1 Announce Type: new Abstract: Synthetic UAV imagery can expand pre-deployment test evidence when task-specific Real data are scarce, but visual plausibility or semantic label alignme
arXiv:2610.18455v1 Announce Type: new Abstract: Rapid advances in artificial intelligence-generated content (AIGC) have significantly promoted using text-to-image (T2I) models to generate images with
arXiv:2610.18053v1 Announce Type: cross Abstract: General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks. Existing benchmarks can
arXiv:2610.18461v1 Announce Type: new Abstract: Video subtitle removal (VSR) is an important video restoration task, yet existing evaluation remains limited by low-resolution datasets, fragmented test
arXiv:2610.18564v1 Announce Type: new Abstract: Aerial map matching provides position observations for navigation, but a position error alone does not determine how an observation should be weighted.
arXiv:2610.15722v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are good at video understanding and code generation, but it remains unclear whether they can recover the physic
arXiv:2610.17057v1 Announce Type: cross Abstract: Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This
We benchmarked Gemini 4 Argon, Muse Wonder, and Muse Spark comprehensively on hard document OCR tasks. They're all pretty good! - Muse Spark and Gemini 4 Argon* are SOTA at their respective price poin
arXiv:2610.13607v1 Announce Type: new Abstract: Negative polarity items (NPIs), such as 'anymore' and 'ever', are grammatical only in select environments, predominantly in the context of negation. Bec
arXiv:2610.06354v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly explored for graph understanding and decision-making, while System One models such as Jev select direc
arXiv:2610.13730v1 Announce Type: new Abstract: Origin-destination (OD) flow forecasting supports urban mobility decisions by predicting the volume and patterns of human movement across a city. Most e
arXiv:2610.13626v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used throughout the software development lifecycle. This growing reliance on LLMs raises concerns about
arXiv:2610.11220v1 Announce Type: new Abstract: Building upon the foundations laid by our previous work, this paper introduces Arena 5.0, the fifth iteration of our framework for robotics social navig
arXiv:2610.11118v1 Announce Type: new Abstract: The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond est
arXiv:2610.12423v1 Announce Type: new Abstract: We present Pumpire, a unified benchmark for evaluating metric point-pair distance estimation capability of both image- and video-level 3D foundation mod
arXiv:2610.10421v1 Announce Type: new Abstract: Goal-oriented Vision-and-Language Navigation (VLN) requires agents to locate and reach targets described in natural language without prescribed routes.
arXiv:2610.09096v1 Announce Type: new Abstract: While established literature underscores the pivotal role of preprocessing in forecasting accuracy, this stage remains largely overlooked in current res
arXiv:2610.09702v1 Announce Type: new Abstract: Ordinary prompt-based editing can cause latent watermark detection to fail without explicitly targeting the watermark. We benchmark eight watermark meth
arXiv:2610.10384v1 Announce Type: new Abstract: Tactile feedback provides embodied agents with physical information beyond visual observations, enabling more reliable interaction with the real world.
arXiv:2610.08813v1 Announce Type: new Abstract: X-ray image-based Radiology Report Generation (RRG) constitutes a critical research direction within medical artificial intelligence, with great potenti
arXiv:2610.08936v1 Announce Type: new Abstract: Robustness of image classification has several benchmarks, but their video counterparts are absent. In video classification temporal dimension introduce
arXiv:2610.06973v1 Announce Type: new Abstract: Melt-pool monitoring is central to qualifying metal additive manufacturing (AM), yet no public event-camera benchmark exists for this domain. Event came
arXiv:2610.06947v1 Announce Type: cross Abstract: Factor mining seeks to discover signals from financial data that predict future asset returns and guide portfolio construction. Automated factor minin