Search: “pretraining”

Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions

model-releasesarxiv-cs-ai

arXiv:2605.29738v1 Announce Type: cross Abstract: Legal NLP benchmarks overwhelmingly evaluate a single language or aggregate tasks that differ fundamentally across jurisdictions, making cross-lingual

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

DGX agent

arXiv:2605.30280v1 Announce Type: cross Abstract: Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented cap

model-releasesarxiv-cs-ai

29 May 2026

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

model-releasesarxiv-cs-cl

arXiv:2605.28827v1 Announce Type: new Abstract: Open Arabic large language models split into two classes: sub-1B multilingual models that treat Arabic as an afterthought (Qwen2.5-0.5B, Falcon-H1-0.5B)

Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets

arXiv:2605.30289v1 Announce Type: new Abstract: Numeric tabular datasets are the dominant data format in scientific practice, yet large language models lack native mechanisms for representing numeric

SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation

arXiv:2605.29655v1 Announce Type: new Abstract: Autoregressive multimodal large language models (MLLMs) enable 3D generation but struggle to scale to high-resolution shapes due to inadequate 3D tokeni

Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods

model-releasesarxiv-cs-cv

arXiv:2601.12500v2 Announce Type: replace Abstract: Counting and tracking dense crowds in large-scale scenes is a highly practical yet challenging problem. Existing methods mostly rely on fixed-camera

VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2605.29605v1 Announce Type: new Abstract: Confidence estimation for Vision-Language-Action (VLA) models is essential for robots to perform manipulation tasks in the open world, providing crucial

Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention

arXiv:2605.29548v1 Announce Type: new Abstract: Larger models learn tasks smaller models do not. What drives this phenomenon? We develop a simple phenomenological argument that power-law scaling alrea

tutorialsarxiv-cs-lg

A Factory-Floor Deployment Case Study of VLA Pipelines for Industrial Packaging Task: Workflow, Failures, and Lessons

arXiv:2605.27461v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies have shown promising manipulation capabilities, yet their practical impact is often limited by the reliability dem

safetyarxiv-cs-ro

A Structural Theory of Position Bias in Transformers

arXiv:2602.16837v2 Announce Type: replace Abstract: Transformer models systematically favor certain token positions, yet the architectural origins of this position bias remain poorly understood. This

Local Ai

An Evolutionary Approach for Designing Stable and Highly Expressible Low-Immunogenicity Therapeutic mRNA Sequences

arXiv:2605.27986v1 Announce Type: new Abstract: Messenger RNA (mRNA) sequences as therapeutics require optimized design to ensure efficient translation, structural stability, and minimal immunogenicit

local-aiarxiv-cs-cl

Applications of temporal graph learning for predicting the dynamics of biological systems

arXiv:2605.28659v1 Announce Type: new Abstract: Biological foundation models have shown strong performance in single-cell representation learning by applying transformer architectures directly to gene

Agents

AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning

arXiv:2605.28287v1 Announce Type: new Abstract: Discovering novel stable molecules without training data remains a grand scientific challenge. Current molecular generative models are trained on large,

agentsarxiv-cs-lg

Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity

model-releasesarxiv-cs-lg

arXiv:2605.28640v1 Announce Type: new Abstract: Efficient inference is critical for long-context language models, where attention computation and KV-cache access dominate the cost. Recent work RAT+, i

Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?

arXiv:2605.27733v1 Announce Type: new Abstract: Training instabilities such as loss spikes are frequently the result of stochastic gradient noise. Because of rare expressions in language training data

COTTA: Context-Aware Transfer Adaptation for Trajectory Prediction in Autonomous Driving

arXiv:2604.00402v2 Announce Type: replace-cross Abstract: Developing robust models to accurately predict the trajectories of surrounding agents is fundamental to autonomous driving safety. However, mo

safetyarxiv-cs-ai

Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies

arXiv:2605.24302v2 Announce Type: replace Abstract: Egocentric action recognition is a challenging task due to erratic camera motion, frequent hand occlusion, and the difficulty of maintaining consist

DebFilter: Eradicating Biases Stashed in Value

arXiv:2605.28167v1 Announce Type: new Abstract: Text-to-image diffusion models, which are theoretically equivalent to score-based generative models, generate images through a multi-step denoising proc

safetyarxiv-cs-cv

Explicit Critic Guidance for Aligning Diffusion Models

arXiv:2605.27736v1 Announce Type: cross Abstract: Online reinforcement learning is becoming increasingly important for aligning diffusion models with non-differentiable objectives. However, existing m

Fine-Tuning Dynamics of In-Context Factual Recall in Transformers

arXiv:2605.27774v1 Announce Type: new Abstract: In-context learning -- performing tasks based on examples given in the prompt -- is an important capability that has emerged in large language models an

Imitation Learning for Robot Assistance in Open Surgery: A Multi-Policy Evaluation on Suture Following

model-releasesarxiv-cs-ro

arXiv:2605.28736v1 Announce Type: new Abstract: This study presents the first evaluation of general-purpose imitation learning for surgeon-robot collaborative assistance in open surgery, targeting sut

Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

arXiv:2605.27583v1 Announce Type: new Abstract: Electrocardiograms (ECGs) are widely used non-invasive measurements of cardiac activity and play a central role in clinical diagnosis. Recent multimodal

Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration

arXiv:2605.28184v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as the standard paradigm for improving reasoning capability of large language models,

Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images

arXiv:2605.28693v1 Announce Type: cross Abstract: Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains h

tutorialsarxiv-cs-ai

Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction

arXiv:2605.27878v1 Announce Type: new Abstract: Large language models produce fluent fiction, yet their creative output is widely seen as flat. We ask where this quality originates in the training and

safetyarxiv-cs-cl

Neural Weight Compression for Language Models

arXiv:2510.11234v3 Announce Type: replace Abstract: Efficient compression of language model weights is increasingly critical as model scale and deployment grow. Yet, most existing methods rely on hand

Local Ai

Outer-Momentum Restarting in High-Dimensional Two-Phase Optimization

arXiv:2605.28585v1 Announce Type: new Abstract: Communication-efficient distributed optimizers such as DiLoCo reduce synchronization costs by letting workers perform many local updates before aggregat

local-aiarxiv-cs-lg

PhAME: Phenotype-Aware Molecular Editing via Latent Diffusion

arXiv:2605.28226v1 Announce Type: new Abstract: Small-molecule drug discovery requires simultaneous optimization of numerous properties of candidate molecules. These properties can be investigated thr

ProgVLA: Progress-Aware Robot Manipulation Skill Learning

model-releasesarxiv-cs-lg

arXiv:2605.28231v1 Announce Type: cross Abstract: We present ProgVLA, a compact vision-language-action (VLA) model designed for reliable robot manipulation under tight compute and memory budgets. The

Agents

RAG-Coding: Enhancing LLM Medical Coding with Structured External Knowledge

arXiv:2605.27377v1 Announce Type: cross Abstract: We present RAG-Coding, an agentic method for automated ICD-10-CM coding. RAG-Coding orchestrates four large language model (LLM) agents and grounds th

agentsarxiv-cs-ai

Routing-Aligned Fine-Tuning for Multilingual Downstream Tasks in Mixture-of-Experts Models

arXiv:2605.28306v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models have emerged as a dominant paradigm for efficient LLM scaling, yet adapting them to non-English downstream tasks remai

safetyarxiv-cs-ai

Soro: A Lightweight Foundation Model and Chatbot for Tajik

model-releasesarxiv-cs-ai

arXiv:2605.27379v1 Announce Type: new Abstract: We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and co

SPRINT: Efficient Spectral Priors for Humanoid Athletic Sprints

arXiv:2605.28549v1 Announce Type: cross Abstract: The pursuit of humanoid athletic sprints is hindered by a scarcity of humanoid-viable kinematic reference data and the inability of existing framework

Super-Resolved Canopy Height Mapping from Sentinel-2 Time Series Using Airborne LiDAR HD Reference Data across Metropolitan France

arXiv:2512.11524v3 Announce Type: replace Abstract: Fine-scale forest monitoring is essential for understanding canopy structure and its dynamics, which are key indicators of carbon stocks, biodiversi

Syllabic-Structure Decoder for Automatic Speech Recognition in Vietnamese

arXiv:2605.27874v1 Announce Type: new Abstract: Most Automatic Speech Recognition (ASR) systems formulate transcription as a prediction problem over orthographic units such as characters, subwords, or

researcharxiv-cs-cl

Text-Only Data Synthesis for Vision Language Model Training

arXiv:2503.22655v2 Announce Type: replace Abstract: Training vision-language models (VLMs) typically requires large-scale, high-quality image-text pairs, but collecting or synthesizing such data is co

researcharxiv-cs-ai

The Abstraction Gap in Vision-Language Causal Reasoning

model-releasesarxiv-cs-cl

arXiv:2605.28779v1 Announce Type: new Abstract: Vision-language models (VLMs) generate fluent causal explanations, but current evaluations cannot distinguish linguistic plausibility from faithful caus

Applications

Toward Semantic-Agnostic and Shape-Aware Vision-Language Segmentation Models

arXiv:2605.28348v1 Announce Type: new Abstract: Vision-language segmentation models have recently achieved strong performance by leveraging high-level semantic object categories expressed in natural l

applicationsarxiv-cs-cv

Transfer learning RGB models to hyperspectral images with trainable tensor decompositions

arXiv:2605.28331v1 Announce Type: new Abstract: Transfer learning makes it possible to use large vision networks on a variety of domains, by specializing their models' general filters to new tasks. Ho

Understanding Generalization and Forgetting in In-Context Continual Learning

model-releasesarxiv-cs-lg

arXiv:2605.28705v1 Announce Type: new Abstract: In-context learning (ICL) derives its power from enabling Large Language Models to adapt to new tasks via prompt-based reasoning alone, entirely bypassi

Hardware

Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation

arXiv:2605.27582v1 Announce Type: cross Abstract: Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environme

hardwarearxiv-cs-cv

UniMaia: Steering Chess Policies with Language for Human-like Play

model-releasesarxiv-cs-ai

arXiv:2605.27767v1 Announce Type: cross Abstract: Recent advances in large language models have enabled natural language to serve as a flexible interface for controlling complex systems, but often at

xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction

arXiv:2503.18893v2 Announce Type: replace Abstract: Long-context Large Language Models (LLMs) enable powerful applications but incur high memory costs due to the key-value states (KV-Cache). Recent st

safetyarxiv-cs-cl

Axial-Centric Cross-Plane Attention for 3D Medical Image Classification

model-releasesarxiv-cs-cv

arXiv:2602.21636v2 Announce Type: replace Abstract: Abridged: Clinicians commonly interpret 3D medical images by examining multiple anatomical planes rather than relying on volumetric views. In clinic

BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma

arXiv:2605.26376v1 Announce Type: cross Abstract: Hepatocellular carcinoma (HCC) is biologically heterogeneous, shaped by the interplay between hepatic functional reserve and tumor-related oncologic f

tutorialsarxiv-cs-ai

Co-folding model guided by structural proteomics

arXiv:2605.26192v1 Announce Type: cross Abstract: Protein structure generative models excel at predicting single protein static structures from sequence, but routinely fail to capture the correct conf

researcharxiv-cs-ai

COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection

model-releasesarxiv-cs-cv

arXiv:2605.27116v1 Announce Type: new Abstract: Open-vocabulary object detection (OVD) has made significant progress, enabling detectors to generalize from seen to unseen categories. However, real-wor

Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in modern Transformers

arXiv:2601.20796v2 Announce Type: replace Abstract: Transformer-based multimodal large language models often exhibit in-context learning (ICL) abilities. Motivated by this phenomenon, we ask: how do t

tutorialsarxiv-cs-cl