Knowledge catalogue
arXiv:2608.19901v1 Announce Type: cross Abstract: Agent Skills extend LLM agents with reusable instruction packages that may also include scripts, resources, and service configuration. This creates a
arXiv:2608.19802v1 Announce Type: new Abstract: LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and t
arXiv:2608.19218v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are increasingly being explored for domain-specific maintenance and prognostics tasks, raising the
arXiv:2608.19231v1 Announce Type: cross Abstract: Estimating large and simulation-intensive discrete choice models (DCMs) requires repeated evaluation of utilities, probabilities, derivatives, and sim
arXiv:2608.19583v1 Announce Type: cross Abstract: Recent studies suggest that video generation models can exhibit certain forms of zero-shot visual reasoning through generated frames. Yet reliable eva
arXiv:2608.18710v1 Announce Type: new Abstract: Recent advances in generative video models have enabled camera-controlled world video generation, allowing models to synthesize videos under user-define
arXiv:2608.18107v1 Announce Type: cross Abstract: We investigate whether large language models (LLMs) systematically discriminate in candidate evaluations based on applicant name ethnicity and/or inst
arXiv:2608.18704v1 Announce Type: cross Abstract: Long-term memory is essential for agents that operate across extended interactions, yet existing memory systems and benchmarks predominantly focus on
arXiv:2608.18131v1 Announce Type: new Abstract: Current safety alignment training for Large Language Models (LLMs) are heavily English-centric. When such safety filters fail for non-English languages,
arXiv:2608.18701v1 Announce Type: new Abstract: Physical interaction quality is central to deformable-object manipulation, yet most benchmarks evaluate task success alone. A policy may complete the ta
arXiv:2608.18280v1 Announce Type: cross Abstract: Background. Advances in agentic systems are simultaneously, and rapidly, saturating benchmarks. Despite this often discussed phenomena, benchmark scor
arXiv:2608.17671v1 Announce Type: cross Abstract: Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools report success rates above 80% on their respective
arXiv:2608.18027v1 Announce Type: new Abstract: Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models' ability to improve through inf
arXiv:2608.17091v1 Announce Type: new Abstract: While publicly available electricity market data presents a valuable resource for forecasting research, the field lacks established benchmark datasets f
arXiv:2608.17522v1 Announce Type: cross Abstract: Persistent shortages in the surgical workforce and inherent limitations of traditional training methods highlight the necessity of automated, data-dri
arXiv:2608.17597v1 Announce Type: cross Abstract: Large language models are increasingly deployed through agent harnesses that manage tools, extensions, persistent state, permissions, and external act
arXiv:2603.10302v3 Announce Type: replace Abstract: A plethora of protein language models have been released in recent years. Yet comparatively little work has addressed how to best sample from them t
arXiv:2608.17600v1 Announce Type: new Abstract: Visual cues are increasingly adopted to guide robot learning, but whether Vision-Language-Action (VLA) models can reliably follow authorized cues while
arXiv:2608.16921v2 Announce Type: cross Abstract: Effective cybersecurity operations require timely and accurate analysis of large-scale heterogeneous security information; however, analysts increasin
arXiv:2608.17450v1 Announce Type: cross Abstract: Learning restricted Boltzmann machines (RBMs) is computationally challenging because it requires expectations whose exact evaluation is generally intr
arXiv:2608.17110v1 Announce Type: new Abstract: Open-vocabulary monocular 3D detectors report strong in-domain performance, but each evaluates under a different protocol, several rely on per-image cat
arXiv:2608.17426v1 Announce Type: cross Abstract: We introduce Semantic Task Completion Video Generation, an outcome-oriented video generation task. Under this formulation, success requires both achie
arXiv:2608.17237v1 Announce Type: cross Abstract: Converting structural framing plans into editable finite-element model drafts remains labor-intensive and prone to transcription error. Existing drawi
arXiv:2608.17795v1 Announce Type: new Abstract: Text-to-SQL systems are commonly evaluated using ground-truth SQL queries or reference execution results, but such supervision is unavailable at inferen
arXiv:2608.17799v1 Announce Type: cross Abstract: Ground-to-Air (G2A) drone detection in medium- and long-wave infrared (MWIR/LWIR) imagery is challenging due to reduced texture information, sensor no
arXiv:2608.14622v1 Announce Type: new Abstract: People are increasingly using large language models (LLMs) to seek advice, including for parenting. Parenting is a critical and socially sensitive domai
arXiv:2507.04491v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are rapidly being integrated into psychological and behavioral research as research tools, evaluation targets, hu
arXiv:2608.15932v1 Announce Type: new Abstract: As state-of-the-art machine translation models saturate standard benchmarks, the field needs more challenging evaluations to distinguish between models
arXiv:2608.16286v1 Announce Type: new Abstract: While various organizations now actively encourage LLM use in classrooms, we still lack rigorous, systematic evaluations of how well these models actual
arXiv:2608.16039v1 Announce Type: cross Abstract: Brain parcellation and classification are typically evaluated in isolation, yet downstream AD detection performance depends on their interaction. We d
arXiv:2604.03904v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often produce confident but incorrect answers, in part because standard evaluation incentives reward guessing ove
arXiv:2608.15669v1 Announce Type: new Abstract: Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecul
arXiv:2608.14552v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated and used in medicine, but clinical usefulness depends on answer accuracy and whether confidence
arXiv:2608.15242v1 Announce Type: new Abstract: When a long-horizon agent execution fails, outcome-level evaluation reveals the unsuccessful result but not where the decisive error entered the traject
arXiv:2407.00890v5 Announce Type: replace-cross Abstract: This paper presents a comparative analysis evaluating the accuracy of Large Language Models (LLMs) against traditional macro time series forec
Online evals are difficult to set up correctly. Even if you know what you want to look for, it's an optimization question of which model to choose, how to tune the prompt, and how to think about the c
arXiv:2608.15131v1 Announce Type: new Abstract: Digital platforms govern by changing rules: rankings, monetization thresholds, moderation standards, verification systems, disclosure requirements, appe
arXiv:2608.16410v1 Announce Type: new Abstract: Combined algorithm selection and hyperparameter optimization (CASH) searches a conditional space in which the selected model determines which hyperparam
arXiv:2608.16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi
arXiv:2607.14345v4 Announce Type: replace-cross Abstract: People use language models for practical questions whose answers are difficult to verify. We show that models exhibit covert value leakage: th
arXiv:2608.16045v1 Announce Type: cross Abstract: LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded fil
arXiv:2608.16370v1 Announce Type: new Abstract: Task completion is the standard metric for evaluating context compression, yet it is incomplete: compression can increase an agent's interaction cost by
arXiv:2608.16801v1 Announce Type: new Abstract: We study how teams of AI coding agents coordinate while solving programming tasks. Current evaluations usually report whether the agents complete the ta
arXiv:2608.15641v1 Announce Type: new Abstract: This paper evaluates Wiktionary as an ethically crowdsourced lexicon for English dialects. We took a two-phase approach, providing an in-depth descripti
arXiv:2603.11799v2 Announce Type: replace Abstract: Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Ca
arXiv:2608.13866v1 Announce Type: cross Abstract: Large language models (LLMs) can generate synthetic training data for text classification, but the quality of generated samples is heterogeneous: some
arXiv:2204.13805v5 Announce Type: replace-cross Abstract: Gender gaps in how scientific work is evaluated are well documented, but their sources remain debated. We ask whether an overlooked factor---t
arXiv:2608.14262v1 Announce Type: new Abstract: Temporal vision-language models (TVLMs) offer a reusable, prompt-based interface for surgical video understanding, yet, their robustness under clinicall
arXiv:2608.14491v1 Announce Type: cross Abstract: Network-level maintenance planning requires repeated evaluations of equilibrium traffic flows under road capacity reductions. While equilibrium traffi
arXiv:2608.14379v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have recently achieved promising performance in robotic manipulation. However, existing benchmarks mainly evaluate
arXiv:2608.14157v1 Announce Type: new Abstract: Mechanical ventilation is a critical life-support intervention, requiring dynamic adjustments to ventilator settings as a patient's condition evolves. W
arXiv:2608.13612v1 Announce Type: new Abstract: Natural-language interfaces to enterprise data must translate underspecified requests into governed, executable behavior while controlling invalid queri
arXiv:2608.13711v1 Announce Type: cross Abstract: Computer-aided detection (CADe) systems for colonoscopy promise to reduce clinical miss rates, yet reliable real-world deployment remains elusive. Thi
OpenAI has shut down the internal team that evaluated whether its AI models could pose “catastrophic risks,” a decision reported by the Financial Times on August 16, 2026. The move was announced via a
arXiv:2608.12368v1 Announce Type: new Abstract: Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs). Yet agreement in final labels does not sh
arXiv:2608.13560v1 Announce Type: cross Abstract: Transforming multimodal sources into condensed and structured media outputs can be fundamentally conceptualized as a long-horizon agentic process cent
arXiv:2608.13441v1 Announce Type: new Abstract: Although multimodal large language models (MLLMs) have shown substantial potential in visual understanding and graphic code generation, editing scientif
arXiv:2608.12448v1 Announce Type: new Abstract: Is it possible to find an objective classification criterion that organizes the complexity of gust-induced loads across many flight conditions? And one
arXiv:2608.13395v1 Announce Type: new Abstract: Reinforcement learning improves autonomous-driving vision-language-action (VLA) models by evaluating trajectories sampled from the current policy. Group
arXiv:2608.13255v1 Announce Type: cross Abstract: Geometry-conditioned multi-view diffusion enables high-quality 3D texture generation, but its repeated per-view denoiser evaluations introduce substan