Search: “multimodal”

Toward Consistent World Models with Multi-Token Prediction and Latent Semantic Enhancement

arXiv:2604.06155v2 Announce Type: replace-cross Abstract: Whether Large Language Models (LLMs) develop coherent internal world models remains a core debate. While conventional Next-Token Prediction (N

safetyarxiv-cs-cl

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Towards Universal Skeleton-Based Action Recognition

DGX agent

arXiv:2604.17013v1 Announce Type: new Abstract: With the development of robotics, skeleton-based action recognition has become increasingly important, as human-robot interaction requires understanding

safetyarxiv-cs-cv

21 Apr 2026

VIDEOP2R: Video Understanding from Perception to Reasoning

arXiv:2511.11113v2 Announce Type: replace Abstract: Reinforcement fine-tuning (RFT), a two-stage framework consisting of supervised fine-tuning (SFT) and reinforcement learning (RL) has shown promisin

safetyarxiv-cs-cv

VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning

arXiv:2601.15724v2 Announce Type: replace Abstract: Long-form video understanding remains a fundamental challenge for current Video Large Language Models. Most existing models rely on static reasoning

local-aiarxiv-cs-cv

Will People Enjoy a Robot Trainer? A Case Study with Snoopie the Pacerbot

arXiv:2604.18331v1 Announce Type: new Abstract: The physicality of exercise makes the role of athletic trainers unique. Their physical presence allows them to guide a student through a motion, demonst

agentsarxiv-cs-ro

ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction

arXiv:2511.01188v2 Announce Type: replace Abstract: The rapid spread of fake news threatens social stability and public trust, highlighting the urgent need for its effective detection. Although large

safetyarxiv-cs-cl

AdaVFM: Adaptive Vision Foundation Models for Edge Intelligence via LLM-Guided Execution

arXiv:2604.15622v1 Announce Type: new Abstract: Language-aligned vision foundation models (VFMs) enable versatile visual understanding for always-on contextual AI, but their deployment on edge devices

local-aiarxiv-cs-cv

Applications

An Empirical Study of Validating Synthetic Data for Text-Based Person Retrieval

arXiv:2503.22171v2 Announce Type: replace Abstract: Data plays a pivotal role in Text-Based Person Retrieval (TBPR) research. Mainstream research paradigm necessitates real-world person images with ma

applicationsarxiv-cs-cv

BioHiCL: Hierarchical Multi-Label Contrastive Learning for Biomedical Retrieval with MeSH Labels

arXiv:2604.15591v1 Announce Type: cross Abstract: Effective biomedical information retrieval requires modeling domain semantics and hierarchical relationships among biomedical texts. Existing biomedic

ChatENV: An Interactive Vision-Language Model for Sensor-Guided Environmental Monitoring and Scenario Simulation

model-releasesarxiv-cs-cv

arXiv:2508.10635v3 Announce Type: replace Abstract: Understanding environmental changes from remote sensing imagery is vital for climate resilience, urban planning, and ecosystem monitoring. Yet, curr

Concept-wise Attention for Fine-grained Concept Bottleneck Models

arXiv:2604.15748v1 Announce Type: new Abstract: Recently impressive performance has been achieved in Concept Bottleneck Models (CBM) by utilizing the image-text alignment learned by a large pre-traine

safetyarxiv-cs-cv

Deep Learning-Enabled Modality Transfer Between Independent Microscopes for High-Throughput Imaging

arXiv:2403.18026v2 Announce Type: replace-cross Abstract: High-throughput biological imaging is often constrained by a trade-off between acquisition speed and image quality. Fast imaging modalities, s

local-aiarxiv-cs-lg

Learning to Look before Learning to Like: Incorporating Human Visual Cognition into Aesthetic Quality Assessment

arXiv:2604.15853v1 Announce Type: new Abstract: Automated Aesthetic Quality Assessment (AQA) treats images primarily as static pixel vectors, aligning predictions with human-rating scores largely thro

safetyarxiv-cs-cv

MFC-RFNet: A Multi-scale Guided Rectified Flow Network for Radar Sequence Prediction

arXiv:2601.03633v2 Announce Type: replace-cross Abstract: Accurate and high-resolution precipitation nowcasting from radar echo sequences is crucial for disaster mitigation and economic planning, yet

MMGait: Towards Multi-Modal Gait Recognition

model-releasesarxiv-cs-cv

arXiv:2604.15979v1 Announce Type: new Abstract: Gait recognition has emerged as a powerful biometric technique for identifying individuals at a distance without requiring user cooperation. Most existi

Multi-View Attention Multiple-Instance Learning Enhanced by LLM Reasoning for Cognitive Distortion Detection

arXiv:2509.17292v3 Announce Type: replace-cross Abstract: Cognitive distortions have been closely linked to mental health disorders, yet their automatic detection remains challenging due to contextual

Applications

Polyglot: Multilingual Style Preserving Speech-Driven Facial Animation

arXiv:2604.16108v1 Announce Type: new Abstract: Speech-Driven Facial Animation (SDFA) has gained significant attention due to its applications in movies, video games, and virtual reality. However, mos

applicationsarxiv-cs-cv

Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking

arXiv:2604.15621v1 Announce Type: cross Abstract: Adaptive Retrieval-Augmented Generation aims to mitigate the interference of extraneous noise by dynamically determining the necessity of retrieving s

Reward Weighted Classifier-Free Guidance as Policy Improvement in Autoregressive Models

arXiv:2604.15577v1 Announce Type: cross Abstract: Consider an auto-regressive model that produces outputs x (e.g., answers to questions, molecules) each of which can be summarized by an attribute vect

Sentiment Analysis of German Sign Language Fairy Tales

arXiv:2604.16138v1 Announce Type: new Abstract: We present a dataset and a model for sentiment analysis of German sign language (DGS) fairy tales. First, we perform sentiment analysis for three levels

researcharxiv-cs-cl

Applications

Sketching the Readout of Large Language Models for Scalable Data Attribution and Valuation

arXiv:2604.16197v1 Announce Type: new Abstract: Data attribution and valuation are critical for understanding data-model synergy for Large Language Models (LLMs), yet existing gradient-based methods s

applicationsarxiv-cs-lg

Targeted Exploration via Unified Entropy Control for Reinforcement Learning

arXiv:2604.14646v2 Announce Type: replace Abstract: Recent advances in reinforcement learning (RL) have improved the reasoning capabilities of large language models (LLMs) and vision-language models (

Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures

arXiv:2604.16042v1 Announce Type: cross Abstract: While Large Language Models (LLMs) have achieved strong performance across many NLP tasks, their opaque internal mechanisms hinder trustworthiness and

Tools

🔬 Training Transformers to solve 95% failure rate of Cancer Trials — Ron Alfa & Daniel Bear, Noetik

Noetik uses transformer neural networks and machine learning to improve cancer drug trial success rates, addressing the historically high failure rate (95%) in clinical development. The approach lever

toolslatent-space

Upload single or multi-view images to ComfyUI’s HY 3D 3.0 nodes for high-fidelity 3D assets—from product displays to game dev, done in minut…

ComfyUI's HY 3D 3.0 nodes enable rapid 3D asset generation from single or multi-view images, producing high-fidelity 3D models suitable for applications including product visualization and game develo

local-aicomfyui--x

Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization

arXiv:2604.16248v1 Announce Type: new Abstract: Image geolocalization has traditionally been addressed through retrieval-based place recognition or geometry-based visual localization pipelines. Recent

local-aiarxiv-cs-cv

At the Beijing half-marathon, several humanoid robots beat human winners by 10+ minutes; a robot made by Honor beat the human world record held by Jacob Kiplimo (Reuters)

Reuters: At the Beijing half-marathon, several humanoid robots beat human winners by 10+ minutes; a robot made by Honor beat the human world record held by Jacob Kiplimo — Dozens of Chinese-made human

agentstechmeme

19 Apr 2026

Video of my opening keynote for the 2026 World Modeling Workshop at Mila, Quebec AI Institute: simple but powerful ways of using world model…

Video of my opening keynote for the 2026 World Modeling Workshop at Mila, Quebec AI Institute: simple but powerful ways of using world models and their latent space (sorry for sound problem 0:20-0:43)

researchdavid-ha--x

19 Apr 2026

Tutorials

CART: Context-Aware Terrain Adaptation using Temporal Sequence Selection for Legged Robots

arXiv:2604.14344v1 Announce Type: new Abstract: Animals in nature combine multiple modalities, such as sight and feel, to perceive terrain and develop an understanding of how to walk on uneven terrain

tutorialsarxiv-cs-ro

ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling

model-releasesarxiv-cs-cv

arXiv:2604.15086v1 Announce Type: cross Abstract: Recent advances in video-to-audio (V2A) generation enable high-quality audio synthesis from visual content, yet achieving robust and fine-grained cont

DEX-Mouse: A Low-cost Portable and Universal Interface with Force Feedback for Data Collection of Dexterous Robotic Hands

arXiv:2604.15013v1 Announce Type: new Abstract: Data-driven dexterous hand manipulation requires large-scale, physically consistent demonstration data. Simulation and video-based methods suffer from s

researcharxiv-cs-ro

Domain-Adaptive Model Merging Across Disconnected Modes

arXiv:2603.05957v2 Announce Type: replace-cross Abstract: Learning across domains is challenging when data cannot be centralized due to privacy or heterogeneity, which limits the ability to train a si

EchoAgent: Towards Reliable Echocardiography Interpretation with 'Eyes','Hands' and 'Minds'

arXiv:2604.05541v2 Announce Type: replace Abstract: Reliable interpretation of echocardiography (Echo) is crucial for assessing cardiac function, which demands clinicians to synchronously orchestrate

agentsarxiv-cs-cv

Explain the Flag: Contextualizing Hate Speech Beyond Censorship

arXiv:2604.14970v1 Announce Type: new Abstract: Hate, derogatory, and offensive speech remains a persistent challenge in online platforms and public discourse. While automated detection systems are wi

researcharxiv-cs-cl

Finetuning-Free Diffusion Model with Adaptive Constraint Guidance for Inorganic Crystal Structure Generation

arXiv:2604.13354v1 Announce Type: cross Abstract: The discovery of inorganic crystal structures with targeted properties is a significant challenge in materials science. Generative models, especially

Flow of Truth: Proactive Temporal Forensics for Image-to-Video Generation

arXiv:2604.15003v1 Announce Type: new Abstract: The rapid rise of image-to-video (I2V) generation enables realistic videos to be created from a single image but also brings new forensic demands. Unlik

HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNet

arXiv:2604.14724v1 Announce Type: new Abstract: Vision State Space Models (SSMs) like Vim, VMamba, and SiMBA rely on complex scanning strategies to adapt sequential SSMs to process 2D images, introduc

Hierarchical Retrieval Augmented Generation for Adversarial Technique Annotation in Cyber Threat Intelligence Text

arXiv:2604.14166v1 Announce Type: new Abstract: Mapping Cyber Threat Intelligence (CTI) text to MITRE ATT&CK technique IDs is a critical task for understanding adversary behaviors and automating threa

safetyarxiv-cs-cl

Improved Multiscale Structural Mapping with Supervertex Vision Transformer for the Detection of Alzheimer's Disease Neurodegeneration

arXiv:2604.14837v1 Announce Type: new Abstract: Alzheimer's disease (AD) confirmation often relies on positron emission tomography (PET) or cerebrospinal fluid (CSF) analysis, which are costly and inv

in 1996 mitzenmacher showed that sampling two backends and picking the better one drops max load exponentially vs. random selection one extr…

in 1996 mitzenmacher showed that sampling two backends and picking the better one drops max load exponentially vs. random selection one extra comparison. that's the whole trick we built pinecone assis

safetypinecone--x

JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation

arXiv:2411.09209v5 Announce Type: replace Abstract: Audio-driven portrait animation has made significant advances with diffusion-based models, improving video quality and lipsync accuracy. However, th

Listening Alone, Understanding Together: Collaborative Context Recovery for Privacy-Aware AI

arXiv:2604.13348v1 Announce Type: new Abstract: We introduce CONCORD, a privacy-aware asynchronous assistant-to-assistant (A2A) framework that leverages collaboration between proactive speech-based AI

Multi-Modal Manipulation via Multi-Modal Policy Consensus

arXiv:2509.23468v3 Announce Type: replace-cross Abstract: Effectively integrating diverse sensory modalities is crucial for robotic manipulation. However, the typical approach of feature concatenation

safetyarxiv-cs-lg

Tutorials

Optimize video semantic search intent with Amazon Nova Model Distillation on Amazon Bedrock

In this post, we show you how to use Model Distillation, a model customization technique on Amazon Bedrock, to transfer routing intelligence from a large teacher model (Amazon Nova Premier) into a muc

tutorialsaws-ml-blog

Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration

arXiv:2503.21970v3 Announce Type: replace Abstract: State-Space Models (SSMs) have attracted considerable attention in Image Restoration (IR) due to their ability to scale linearly sequence length whi

Sharing my current setup to run Qwen3.6 locally in a good agentic setup (Pi + llama.cpp). Should give you a good overview of how good local …

model-releasesclem-delangue--x

Sharing my current setup to run Qwen3.6 locally in a good agentic setup (Pi + llama.cpp). Should give you a good overview of how good local agents are today: # Start llama.cpp server: llama-server -hf

TableNet A Large-Scale Table Dataset with LLM-Powered Autonomous

arXiv:2604.13041v1 Announce Type: cross Abstract: Table Structure Recognition (TSR) requires the logical reasoning ability of large language models (LLMs) to handle complex table layouts, but current

agentsarxiv-cs-ai

Time-RA: Towards Time Series Reasoning for Anomaly Diagnosis with LLM Feedback

model-releasesarxiv-cs-lg

arXiv:2507.15066v5 Announce Type: replace Abstract: Time series anomaly detection (TSAD) has traditionally focused on binary classification and often lacks the fine-grained categorization and explanat