Search: “evaluation”

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

arXiv:2610.14626v1 Announce Type: new Abstract: Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sa

arXiv:2610.15164v1 Announce Type: new Abstract: This paper studies the backbone used by metrics for evaluating the quality of generated images. Although generative models have advanced rapidly, some o

arXiv:2610.16826v1 Announce Type: cross Abstract: Deepfake speech detectors are typically evaluated as binary classifiers on fixed benchmarks, but deployed systems face evolving attacks, codecs, speak

arXiv:2610.15049v1 Announce Type: new Abstract: This study aims to evaluate and compare convolutional neural network (CNN) architectures, specifically AlexNet, Inception, LeNet, VGG16, and MobileNet,

arXiv:2610.15711v1 Announce Type: new Abstract: Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA F

The reality for most organizations is that data lives everywhere. As engineering teams expand and adopt specialized platforms for their workloads, enterprise data naturally disperses across multiple c

Last week we announced that Gemini Enterprise has evolved to make it even simpler to get work done, and introduced the Gemini agent, your new universal agent for work. It's always on and ready to answ

arXiv:2610.13703v1 Announce Type: new Abstract: The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming t

arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of K actions, requiring O(K) score evaluations p

arXiv:2610.13928v1 Announce Type: new Abstract: Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilit

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation a

arXiv:2610.12849v1 Announce Type: cross Abstract: Current software engineering agents are evaluated predominantly inside isolated sandboxes against synchronous local tests. However, deploying features

arXiv:2610.12694v1 Announce Type: new Abstract: Molecular machine learning evaluates out-of-distribution performance with structure disjoint splits, a construction rule fixed once at the scale its lib

arXiv:2610.12960v1 Announce Type: new Abstract: Matching repair after utility changes can reuse valid observations, but new evaluations must respect matching capacity and may return out of order. We s

arXiv:2610.13116v1 Announce Type: new Abstract: As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce s

arXiv:2610.12669v1 Announce Type: cross Abstract: Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. Th

arXiv:2610.13236v1 Announce Type: cross Abstract: Recent advances have significantly improved the performance of Abstract Meaning Representation (AMR) parsers, yet determining when their outputs can b

Agents are only as capable as the context they can reach. An agent with reliable access to the right information makes decisions grounded in fact; one without it guesses. And the work agents produce i

arXiv:2610.12881v1 Announce Type: new Abstract: Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world model

arXiv:2610.13790v1 Announce Type: cross Abstract: How well can vision-language models (VLMs) reason about what is missing from an image? We study this question with OccludeBench, a manually curated be

arXiv:2610.13550v1 Announce Type: new Abstract: Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions. However, little work investigates how thi

arXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue tha

arXiv:2610.13036v1 Announce Type: new Abstract: Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure

arXiv:2610.13247v1 Announce Type: cross Abstract: Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validat

Introducing Multi-Agent Self-Supervision (MASS): Recursive self-improvement through collective intelligence Blog: https://pub.sakana.ai/mass/ As AI systems tackle increasingly open-ended research prob

arXiv:2610.11354v1 Announce Type: new Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in d

arXiv:2610.11362v1 Announce Type: cross Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference c

arXiv:2610.12190v1 Announce Type: new Abstract: As claims about recursive self-improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models

arXiv:2610.11539v1 Announce Type: cross Abstract: As leading LLMs improve on capability evaluations, their limitations in producing creative outputs on design tasks remain insufficiently characterised

arXiv:2610.11012v1 Announce Type: new Abstract: Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. Howeve

arXiv:2610.12076v1 Announce Type: new Abstract: Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the

arXiv:2610.12235v1 Announce Type: new Abstract: AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet thei

arXiv:2610.12379v1 Announce Type: new Abstract: Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The extbf{cond

arXiv:2610.09484v2 Announce Type: replace Abstract: Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and diffic

arXiv:2610.11966v1 Announce Type: new Abstract: Research idea innovation is a fundamental engine of scientific progress, yet it remains difficult to generate and evaluate in a scalable and controllabl

arXiv:2610.11118v1 Announce Type: new Abstract: The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond est

arXiv:2610.11015v1 Announce Type: new Abstract: As voice agents gain more popularity commercially, the user simulators used to evaluate the deployed agents are also being developed to include more rea

arXiv:2610.11183v1 Announce Type: cross Abstract: Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave co

arXiv:2609.32123v2 Announce Type: replace Abstract: Time-series diagnostic systems rarely rely on retrieving relevant historical cases, and when they do, retrieval is evaluated only indirectly through

arXiv:2610.12343v1 Announce Type: new Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, p

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal

arXiv:2610.10560v1 Announce Type: cross Abstract: AI foundation models pretrained on weather and climate data are increasingly fine-tuned to Earth science tasks well beyond weather forecasting. Their

arXiv:2610.10637v1 Announce Type: new Abstract: Hair stroking is common in daily grooming and personal care, and is also widely used in hair-product evaluation, motivating robots with similar physical

arXiv:2610.11392v1 Announce Type: new Abstract: System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generati

arXiv:2610.11674v1 Announce Type: new Abstract: Automated video surveillance analysis has become a critical component of intelligence infrastructures and Law Enforcement agencies. Traditional systems