Knowledge catalogue
arXiv:2512.22278v2 Announce Type: replace Abstract: The growing demand for prenatal ultrasound imaging has intensified a global shortage of trained sonographers, creating barriers to essential fetal h
arXiv:2604.16054v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved impressive progress on vision language benchmarks, yet their capacity for visual cognitive and
arXiv:2604.16170v1 Announce Type: new Abstract: We introduce neuralCAD-Edit, the first benchmark for editing 3D CAD models collected from expert CAD engineers. Instead of text conditioning as in prior
arXiv:2604.15652v1 Announce Type: new Abstract: Open-vocabulary remote sensing image segmentation (OVRSIS) remains underexplored due to fragmented datasets, limited training diversity, and the lack of
arXiv:2512.14554v5 Announce Type: replace-cross Abstract: The rapid advancement of large language models (LLMs) has enabled new possibilities for applying artificial intelligence within the legal doma
Localmaxxing.com is a platform in private testing phase with plans for public release the same week this announcement was made, designed to simplify benchmark sharing by eliminating the need for users
arXiv:2603.04738v2 Announce Type: replace Abstract: Instruction-following is a foundational capability of large language models (LLMs), with its improvement hinging on scalable and accurate feedback f
arXiv:2505.20122v2 Announce Type: replace Abstract: This paper introduces MEBench, a novel benchmark for evaluating mutual exclusivity (ME) bias, a cognitive phenomenon observed in children during wor
A researcher on r/MachineLearning built a political benchmark to evaluate how various LLMs handle sensitive geopolitical and politically contentious questions. Key findings include that Kimi K2 (Moons
arXiv:2604.13416v1 Announce Type: new Abstract: Advances in radiance fields have enabled photorealistic novel view synthesis. In several domains, large-scale real-world datasets have been developed to
arXiv:2505.19662v3 Announce Type: replace-cross Abstract: This paper introduces FieldWorkArena, a benchmark for agentic AI targeting real-world field work. With the recent increase in demand for agent
arXiv:2604.13531v1 Announce Type: cross Abstract: Graphical User Interface (GUI) agents show strong capabilities for automating web tasks, but existing interactive benchmarks primarily target benign,
arXiv:2512.20798v4 Announce Type: replace Abstract: As autonomous AI agents are deployed in high-stakes environments, ensuring their safety has become a paramount concern. Existing safety benchmarks p
arXiv:2604.12659v1 Announce Type: cross Abstract: Vision-language models(VLMs) are increasingly applied to visual stock price forecasting, yet existing benchmarks inadequately evaluate their understan
arXiv:2604.12744v1 Announce Type: new Abstract: While multilingual language models promise to bring the benefits of LLMs to speakers of many languages, gold-standard evaluation benchmarks in most lang
arXiv:2604.10981v1 Announce Type: new Abstract: ATANT v1.0 (arXiv:2604.06710) defined continuity as a system property with 7 required properties and introduced a 10-checkpoint, LLM-free evaluation met
arXiv:2604.10736v1 Announce Type: new Abstract: No open Irish-specific benchmark compares end-user ASR systems under a shared Irish-aware evaluation protocol. To solve this, we release BlasBench, an o
arXiv:2604.11174v1 Announce Type: cross Abstract: Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluatio
arXiv:2604.09907v1 Announce Type: cross Abstract: To improve crop genetics, high-throughput, effective and comprehensive phenotyping is a critical prerequisite. While such tasks were traditionally per
arXiv:2604.09587v1 Announce Type: new Abstract: Mobile agents can autonomously complete user-assigned tasks through GUI interactions. However, existing mainstream evaluation benchmarks, such as Androi
arXiv:2511.17925v3 Announce Type: replace-cross Abstract: Recent advances in whole-body robot control have enabled humanoid and legged robots to perform increasingly agile and coordinated motions. How
arXiv:2604.09985v1 Announce Type: new Abstract: Video Camouflaged Object Detection (VCOD) is currently constrained by the scarcity of challenging benchmarks and the limited robustness of models agains
Grok continues to lead global benchmarks: • #1 in AA Omniscience (lowest hallucination rate) • #1 in IFBench performance • #1 on BridgeBench Reasoning • #1 on BridgeBench Speed • #1 on BridgeBench Low
arXiv:2604.09408v1 Announce Type: new Abstract: Frontier coding agents solve complex tasks when given complete context but collapse when specifications are incomplete or ambiguous. The bottleneck is n
arXiv:2604.08884v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have made significant strides in natural image understanding, their ability to perceive and reason over
A Reddit user on r/ollama conducted a hands-on benchmark comparing Gemma4:e4b (Google's compact ~4.5B effective-parameter edge model) against Gemma3:27B, GPT-4o-mini, and Gemini 2.5 Flash, all run or
A Reddit post in the r/ollama community seeking volunteers with diverse hardware setups to participate in a collaborative effort to benchmark the **behavioral reliability** of locally-run large langua
arXiv:2604.08991v1 Announce Type: cross Abstract: Small object-centric spatial understanding in indoor videos remains a significant challenge for multimodal large language models (MLLMs), despite its
arXiv:2604.09048v1 Announce Type: cross Abstract: While the large energy consumption of Large Language Models (LLMs) is recognized by the community, system operators lack guidance for energy-efficient
arXiv:2604.07749v1 Announce Type: new Abstract: Large language models (LLMs) can shift their answers under pressure in ways that reflect accommodation rather than reasoning. Prior work on sycophancy h
arXiv:2604.08535v1 Announce Type: cross Abstract: Generalization under distribution shift remains a central bottleneck for closed-loop autonomous driving. Although simulators like CARLA enable safe an
arXiv:2512.23994v3 Announce Type: replace-cross Abstract: Text-to-audio-video (T2AV) generation is central to applications such as filmmaking and world modeling. However, current models often fail to
arXiv:2604.08340v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) have achieved remarkable progress in static visual understanding, their deployment in complex 3D embodied environmen
arXiv:2602.22683v2 Announce Type: replace Abstract: The rapid advancement of AI-powered smart glasses-one of the hottest wearable devices-has unlocked new frontiers for multimodal interaction, with Vi
arXiv:2604.06264v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled molecular reasoning for property prediction. However, toxicity arises from complex biolog
The chart says GLM-5.1 scored 54.9 on coding benchmarks. Three points behind Claude Opus 4.6. Interesting but not the story. The story is what trained it. Zero Nvidia GPUs. 100,000 Huawei Ascend 910B
🚀 DeepSeek-V4.1-Flash-1017 is here! Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our
Some strong embeddings models have been released recently. Ours is still better. We benchmarked Embed 5 Fast against Google's EmbeddingGemma 2 across 163 datasets using RCP-nDCG@10. Embed 5 Fast leads
arXiv:2610.21654v1 Announce Type: new Abstract: This paper presents the Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years, designed to b
arXiv:2610.21412v1 Announce Type: new Abstract: Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large
Mistral AI announced in mid‑October 2026 that its new model, Mistral Large 4, achieved high scores on Ulam’s private STEM benchmark (STEMBench), particularly excelling in engineering and astronomy cat
arXiv:2610.21659v1 Announce Type: cross Abstract: Corporate financial disclosures often unfold over time, with announcements, prepared remarks, and discussion spanning multiple high-frequency return i
arXiv:2610.14700v1 Announce Type: new Abstract: Most text-to-SQL benchmarks provide the target database. In a large data catalog, however, choosing the sources is part of the problem: an agent must fi
arXiv:2610.19564v1 Announce Type: new Abstract: A spatial answer depends on the reference frame specified in language and its supporting visual evidence. Item accuracy alone obscures whether correctne
Hermes Agent is the best place to use the new @NVIDIAAI Nemotron 3.5 Super VL and Bolt models. NVIDIA benchmarked Super VL on Hermes Bench with a score of 63.44, and compared to similar local models i
I calculated KLD and perplexity for Qwen3.8 27B models that fit 16GB VRAM: See comments for the benchmarks. Reddit filters keeps deleting the post. Conclusion: q5_1/q5_1 is almost always better than q
arXiv:2610.19159v1 Announce Type: cross Abstract: Can AI agents carry out research on open scientific problems? We introduce Into the Unknown, a benchmark that provides an agent with a science paper a
Jeffrey Kampman / Tom's Hardware: Nvidia's RTX Spark Superchip in the Surface Laptop Ultra outperforms Apple's M5 Max MacBook Pro by ~25% in agentic AI completion benchmarks using Qwen models — Laying
arXiv:2601.08689v3 Announce Type: replace Abstract: Large Language Models (LLMs) have shown strong capabilities in finance-oriented question answering, yet their evaluation on professional quantitativ
arXiv:2610.16942v1 Announce Type: cross Abstract: Large audio-language models (LALMs) are fast becoming a general interface to sound, yet how far their broad pretraining carries into specialized domai
arXiv:2610.15179v1 Announce Type: cross Abstract: Large language models are increasingly deployed in contexts requiring adherence to ethical and legal norms, yet their alignment with internationally r
arXiv:2610.16577v1 Announce Type: new Abstract: Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes pro
arXiv:2610.15345v1 Announce Type: new Abstract: High-quality coding problems are expensive to collect. Benchmarks such as SWE-bench rely on substantial human curation, while synthetic alternatives typ
arXiv:2610.17794v1 Announce Type: new Abstract: Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external condition
arXiv:2610.18843v1 Announce Type: new Abstract: Robot policies are expected to generalize across deployment environments, including variations in the robot itself: cameras are bumped and remounted, ca
arXiv:2602.05382v2 Announce Type: replace Abstract: Vision-language model (VLM) errors on visual reasoning may come from missing visual structure, from mistakes after that structure is extracted, or f
arXiv:2610.12669v1 Announce Type: cross Abstract: Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. Th
🇫🇷 Le Chonk joins the sudo L7 leaderboard 🇫🇷 We just added @MistralAI's Mistral Large 4 to sudo L7, our benchmark for staff-level software engineering. Mistral Large 4 scores 13.3%, ahead of Muse Spar
arXiv:2610.12561v1 Announce Type: new Abstract: Memory allows large language model (LLM) agents to use past experience, often stored as textual records, when responding to requests. For embodied agent
arXiv:2610.11065v1 Announce Type: new Abstract: Automating city-scale wireless deployment remains challenging under complex urban propagation and network-wide interference. We introduce extbf{CityDepl