Search: “gemini”

DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories

arXiv:2604.20443v1 Announce Type: cross Abstract: Large Language Models (LLMs) have been shown to possess Theory of Mind (ToM) abilities. However, it remains unclear whether this stems from robust rea

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Do Small Language Models Know When They're Wrong? Confidence-Based Cascade Scoring for Educational Assessment

DGX agent

arXiv:2604.19781v1 Announce Type: cross Abstract: Automated scoring of student work at scale requires balancing accuracy against cost and latency. In 'cascade' systems, small language models (LMs) han

model-releasesarxiv-cs-ai

23 Apr 2026

Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

arXiv:2604.20168v1 Announce Type: new Abstract: This paper presents the Duluth approach to SemEval-2026 Task 6 on CLARITY: Unmasking Political Question Evasions. We address Task 1 (clarity-level class

Early-Stage Product Line Validation Using LLMs: A Study on Semi-Formal Blueprint Analysis

arXiv:2604.20523v1 Announce Type: cross Abstract: We study whether Large Language Models (LLMs) can perform feature model analysis operations (AOs) directly on semi-formal textual blueprints, i.e., co

OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model

arXiv:2604.20806v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) have made substantial advances in reasoning tasks at the Olympiad level. Nevertheless, current Olympiad-level mul

SciCoQA: Quality Assurance for Scientific Paper--Code Alignment

arXiv:2601.12910v3 Announce Type: replace-cross Abstract: Discrepancies between scientific papers and their code undermine reproducibility, a concern that grows as automated research agents scale scie

ThermoQA: A Three-Tier Benchmark for Evaluating Thermodynamic Reasoning in Large Language Models

arXiv:2604.19758v1 Announce Type: new Abstract: We present ThermoQA, a benchmark of 293 open-ended engineering thermodynamics problems in three tiers: property lookups (110 Q), component analysis (101

Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast India

arXiv:2604.19778v1 Announce Type: new Abstract: We present KokborokMT, a high-quality neural machine translation (NMT) system for Kokborok (ISO 639-3), a Tibeto-Burman language spoken primarily in Tri

IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text

arXiv:2604.19298v1 Announce Type: cross Abstract: We introduce IndiaFinBench, to our knowledge the first publicly available evaluation benchmark for assessing large language model (LLM) performance on

Protecting Bystander Privacy via Selective Hearing in Audio LLMs

arXiv:2512.06380v3 Announce Type: replace-cross Abstract: Audio Large language models (LLMs) are increasingly deployed in the real world, where they inevitably capture speech from unintended nearby by

SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

arXiv:2604.19638v1 Announce Type: new Abstract: Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address saf

Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks

arXiv:2604.19697v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have shown promising reasoning abilities, yet evaluating their performance in specialized domains remains chall

Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis

arXiv:2604.16729v1 Announce Type: new Abstract: State-of-the-art large language models (LLMs) show high performance in general visual question answering. However, a fundamental limitation remains: cur

BengaliMoralBench: A Benchmark for Auditing Moral Reasoning in Large Language Models within Bengali Language and Culture

arXiv:2511.03180v2 Announce Type: replace Abstract: As multilingual Large Language Models (LLMs) gain traction across South Asia, their alignment with local ethical norms, particularly for Bengali, sp

Bridging the Reasoning Gap in Vietnamese with Small Language Models via Test-Time Scaling

arXiv:2604.17794v1 Announce Type: new Abstract: The democratization of ubiquitous AI hinges on deploying sophisticated reasoning capabilities on resource-constrained devices. However, Small Language M

CBRS: Cognitive Blood Request System with Bilingual Dataset and Dual-Layer Filtering for Multi-Platform Social Streams

arXiv:2604.16665v1 Announce Type: new Abstract: Urgent blood donation seeking posts and messages on social media often go unnoticed due to the overwhelming volume of daily communications. Traditional

Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs

arXiv:2510.11288v4 Announce Type: replace Abstract: Recent work has shown that narrow finetuning can produce broadly misaligned LLMs, a phenomenon termed emergent misalignment (EM). While concerning,

Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models

arXiv:2604.16980v1 Announce Type: new Abstract: Background: Large language models (LLMs) are increasingly proposed for diagnostic support, but few evaluations use real-world multimodal inpatient data,

Evaluating Tool-Using Language Agents: Judge Reliability, Propagation Cascades, and Runtime Mitigation in AgentProp-Bench

arXiv:2604.16706v1 Announce Type: cross Abstract: Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, but this assumption has rarely been validated a

FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings

arXiv:2604.18109v1 Announce Type: new Abstract: This paper presents factorized linear projection (FLiP) models for understanding pretrained sentence embedding spaces. We train FLiP models to recover t

Frankentext: Stitching random text fragments into long-form narratives

arXiv:2505.18128v4 Announce Type: replace Abstract: We introduce Frankentexts, a long-form narrative generation paradigm that treats an LLM as a composer of existing texts rather than as an author. Gi

From Handwriting to Structured Data: Benchmarking AI Digitisation of Handwritten Forms

arXiv:2604.16504v1 Announce Type: new Abstract: Manual digitisation of structured handwritten documents is slow and costly. We benchmark 17 leading frontier multi-modal large language models and open-

GeoRC: A Benchmark for Geolocation Reasoning Chains

arXiv:2601.21278v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) are good at recognizing the global location of a photograph -- their geolocation prediction accuracy rivals the

MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

arXiv:2604.18584v1 Announce Type: cross Abstract: Mathematical problem solving remains a challenging test of reasoning for large language and multimodal models, yet existing benchmarks are limited in

Medical thinking with multiple images

arXiv:2604.16506v1 Announce Type: cross Abstract: Large language models perform well on many medical QA benchmarks, but real clinical reasoning often requires integrating evidence across multiple imag

MMErroR: A Benchmark for Erroneous Reasoning in Vision-Language Models

arXiv:2601.03331v2 Announce Type: replace Abstract: Recent advances in Vision-Language Models (VLMs) have improved performance in multi-modal learning, raising the question of whether these models tru

Penny Wise, Pixel Foolish: Bypassing Price Constraints in Multimodal Agents via Visual Adversarial Perturbations

arXiv:2604.16515v1 Announce Type: new Abstract: The rapid proliferation of Multimodal Large Language Models (MLLMs) has enabled mobile agents to execute high-stakes financial transactions, but their a

QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning

arXiv:2604.16396v1 Announce Type: new Abstract: Islamic inheritance law (ilm al-mawar{i}th) presents a challenging domain for evaluating large language models' structured reasoning capabilities, requi

RosettaSearch: Multi-Objective Inference-Time Search for Protein Sequence Design

arXiv:2604.17175v1 Announce Type: new Abstract: We introduce RosettaSearch, an inference-time multi-objective optimization approach for protein sequence optimization. We use large language models (LLM

TSVer: A Benchmark for Fact Verification Against Time-Series Evidence

arXiv:2511.01101v2 Announce Type: replace Abstract: Reasoning over temporal and numerical data, such as time series, is a crucial aspect of fact-checking. While many systems have recently been develop

AISysRev -- LLM-based Tool for Title-abstract Screening

arXiv:2510.06708v3 Announce Type: replace-cross Abstract: Conducting systematic reviews is laborious. In the screening or study selection phase, the number of papers can be overwhelming. Recent resear

ASMR-Bench: Auditing for Sabotage in ML Research

arXiv:2604.16286v1 Announce Type: new Abstract: As AI systems are increasingly used to conduct research autonomously, misaligned systems could introduce subtle flaws that produce misleading results wh

Characterising LLM-Generated Competency Questions: a Cross-Domain Empirical Study using Open and Closed Models

arXiv:2604.16258v1 Announce Type: new Abstract: Competency Questions (CQs) are a cornerstone of requirement elicitation in ontology engineering. CQs represent requirements as a set of natural language

ChatENV: An Interactive Vision-Language Model for Sensor-Guided Environmental Monitoring and Scenario Simulation

arXiv:2508.10635v3 Announce Type: replace Abstract: Understanding environmental changes from remote sensing imagery is vital for climate resilience, urban planning, and ecosystem monitoring. Yet, curr

Consistency Analysis of Sentiment Predictions using Syntactic & Semantic Context Assessment Summarization (SSAS)

arXiv:2604.15547v1 Announce Type: cross Abstract: The fundamental challenge of using Large Language Models (LLMs) for reliable, enterprise-grade analytics, such as sentiment prediction, is the conflic

Facial-Expression-Aware Prompting for Empathetic LLM Tutoring

arXiv:2604.15336v1 Announce Type: cross Abstract: Large language models (LLMs) enable increasingly capable tutoring-style conversational agents, yet effective tutoring requires sensitivity to learners

From Benchmarking to Reasoning: A Dual-Aspect, Large-Scale Evaluation of LLMs on Vietnamese Legal Text

arXiv:2604.16270v1 Announce Type: cross Abstract: The complexity of Vietnam's legal texts presents a significant barrier to public access to justice. While Large Language Models offer a promising solu

Intelligent Healthcare Imaging Platform: A VLM-Based Framework for Automated Medical Image Analysis and Clinical Report Generation

arXiv:2509.13590v3 Announce Type: replace-cross Abstract: The rapid advancement of artificial intelligence (AI) in healthcare imaging has revolutionized diagnostic medicine and clinical decision-makin

LLMs Corrupt Your Documents When You Delegate

arXiv:2604.15597v1 Announce Type: new Abstract: Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding

No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus

arXiv:2604.16275v1 Announce Type: new Abstract: This paper explores the response of Large Language Models (LLMs) to user prompts with different degrees of politeness and impoliteness. The Politeness T

QuantSightBench: Evaluating LLM Quantitative Forecasting with Prediction Intervals

arXiv:2604.15859v1 Announce Type: cross Abstract: Forecasting has become a natural benchmark for reasoning under uncertainty. Yet existing evaluations of large language models remain limited to judgme

Qwen3.5-Omni Technical Report

arXiv:2604.15804v1 Announce Type: new Abstract: In this work, we present Qwen3.5-Omni, the latest advancement in the Qwen-Omni model family. Representing a significant evolution over its predecessor,

RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees

arXiv:2604.15736v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) excel at generic video understanding, their ability to support specialized, rule-grounded decision-maki

Assessment Design in the AI Era: A Method for Identifying Items Functioning Differentially for Humans and Chatbots

arXiv:2603.23682v2 Announce Type: replace-cross Abstract: The rapid adoption of large language models (LLMs) in education raises profound challenges for assessment design. To adapt assessments to the

AutoRAN: Automated Hijacking of Safety Reasoning in Large Reasoning Models

arXiv:2505.10846v3 Announce Type: replace Abstract: This paper presents AutoRAN, the first framework to automate the hijacking of internal safety reasoning in large reasoning models (LRMs). At its cor

IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation

arXiv:2511.01014v3 Announce Type: replace Abstract: Instruction-following is a fundamental ability of Large Language Models (LLMs), requiring their generated outputs to follow multiple constraints imp

PolyBench: Benchmarking LLM Forecasting and Trading Capabilities on Live Prediction Market Data

arXiv:2604.14199v1 Announce Type: cross Abstract: Predicting real-world events from live market signals demands systems that fuse qualitative news with quantitative order-book dynamics under strict te

Right at My Level: A Unified Multilingual Framework for Proficiency-Aware Text Simplification

arXiv:2604.05302v2 Announce Type: replace Abstract: Text simplification supports second language (L2) learning by providing comprehensible input, consistent with the Input Hypothesis. However, constru