Search: “multimodal”

applicationsyoshua-bengio--x

A consequence of how frontier models are trained is motivated reasoning, a phenomenon well studied in humans and discussed in this podcast from @PalisadeAI. Our thoughts, beliefs and reasonings tend t

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Framework

DGX agent

arXiv:2608.11891v1 Announce Type: cross Abstract: Governments increasingly fund indigenous foundation models to strengthen national AI capability, digital sovereignty, and multilingual computing. Asse

model-releasesarxiv-cs-ai

13 Aug 2026

Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence

arXiv:2608.12290v1 Announce Type: cross Abstract: Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and rel

CookVoice: Unified Framework for Style Controllable Multi-Modal Human Voice Generation

arXiv:2608.11590v1 Announce Type: cross Abstract: Human voice generation has made rapid progress in speech generation, singing voice generation, voice cloning, and voice editing. However, most existin

safetyarxiv-cs-lg

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning

arXiv:2608.11260v1 Announce Type: new Abstract: Video Anomaly Detection (VAD) aims to identify anomalous events and localize their temporal intervals. Existing approaches exhibit a 'when-what' dissoci

JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis

arXiv:2608.11741v1 Announce Type: cross Abstract: The scholarly exegesis of ancient Chinese characters demands integrating visual observation, linguistic analysis, and historical context. However, exi

Minimax-H3 can generate 42s videos natively on an RTX Pro 6000 in 80 minutes

model-releasesr-stablediffusion

The maximum frame count allowed by the native 'MiniMax H3 Reference to Video' node technically is 1008, even if that's way over the training range of the model, which is 324 frames. But why not try? S

Agents

Multi-Agent Target-Existence Verification and Learned Mask Geometry Refinement: Winning Report of the MeViS-Text Track at the 8th LSVOS Challenge 2026

arXiv:2608.11458v1 Announce Type: new Abstract: We present the first-place solution to the MeViS-Text track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge 2026: referring video obj

agentsarxiv-cs-cv

RoboHarness: A Memory-Augmented Policy Harness for Vision-Language-Action Model Robustness via In-Context Adaptation

model-releasesarxiv-cs-ro

arXiv:2603.24060v3 Announce Type: replace Abstract: Despite the promise of Vision-Language-Action (VLA) models as generalist robotic controllers, their robustness against perceptual noise and environm

SenseNova-Vision: a 7B open model that does segmentation, depth, detection, OCR, and 3D reconstruction with no task-specific heads

model-releasesr-localllama

Stumbled across this new vision model, SenseNova-Vision. It's a 7B MoT model, Apache 2.0 license, which is cool. The main idea is it treats pretty much all computer vision stuff as just one generation

Local Ai

Socioduality: A Relational Process Framework for Human-AI Interaction

arXiv:2608.11322v1 Announce Type: cross Abstract: Human-AI research often evaluates individual capabilities, combined performance, or final outputs, but these approaches do not preserve how one party'

local-aiarxiv-cs-ai

Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training

arXiv:2604.07037v2 Announce Type: replace-cross Abstract: Accelerator-based neutrino physics is entering an energy-frontier regime in which interactions reach the TeV scale and produce exceptionally d

Capturing Uncertainty in Human Motion for Representation Learning in Soccer

arXiv:2608.11203v1 Announce Type: new Abstract: This paper presents a self-supervised representation learning framework for understanding 3D skeleton-based human motion in soccer, using future motion

ComBodied Agents: a New Paradigm of Human-Centric Agentic AI

arXiv:2608.10915v1 Announce Type: new Abstract: After an older adult misses a medication dose, a software agent can send another reminder and an embodied agent can bring the medication. Yet neither ex

Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets

model-releasesarxiv-cs-lg

arXiv:2608.10096v1 Announce Type: cross Abstract: Modern data science increasingly gives rise to hypothesis-testing problems that are not naturally formulated in terms of parameters within prespecifie

DocLang: a markup language for LLMs

IBM has introduced **DocLang**, a new markup language specifically designed for large‑language models (LLMs). Developed by lead researcher Peter Staar and IBM’s document‑parsing team, DocLang aims to

researchibm-research

Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models

arXiv:2608.10525v1 Announce Type: cross Abstract: Historical context integration presents a fundamental challenge for Vision-Language Models (VLMs) in sequential decision-making tasks. Current VLMs pr

researcharxiv-cs-ai

Agents

GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning

arXiv:2505.23399v2 Announce Type: replace Abstract: We propose GAM-Agent, a game-theoretic multi-agent framework for enhancing vision-language reasoning. Unlike prior single-agent or monolithic models

agentsarxiv-cs-ai

GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids

arXiv:2603.25467v3 Announce Type: replace Abstract: Vision-Language Models (VLMs) are powerful open-set reasoners, yet their direct use as anomaly detectors in video surveillance is fragile: without c

Multi-Level Evidence Aggregation for Robust Facial Phenotype Retrieval in Rare Genetic Disorder Prioritization

arXiv:2608.11037v1 Announce Type: new Abstract: AI-assisted facial phenotyping supports rare genetic disorder prioritization by retrieving visually similar diagnosed cases from facial image reference

Applications

Real-World Cooperative Bimanual Dexterous Grasp of Large Objects from Single-View Observations

arXiv:2608.10383v1 Announce Type: new Abstract: Bimanual dexterous grasping of large objects is a critical challenge in robotic manipulation. However, most existing studies focus on sequential manipul

applicationsarxiv-cs-ro

ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation

arXiv:2608.10905v1 Announce Type: new Abstract: On-policy distillation (OPD) applies token-level teacher supervision to student-generated trajectories, but this supervision is not always reliable. Exi

safetyarxiv-cs-lg

SAR2Agri: Learning SAR Intensity Representations for Agricultural Monitoring

model-releasesarxiv-cs-cv

arXiv:2608.11142v1 Announce Type: new Abstract: Agricultural monitoring faces unique challenges, arising from the landscape's complex temporal, phenological, and climate dynamics, yet monitoring them

SkillLens: Visual Skill Cards for Retrieval-Augmented GUI Action Prediction and On-Policy Distillation

arXiv:2608.10775v1 Announce Type: new Abstract: Computer-using agents can perceive rich software interfaces, yet their decisions often lack visual procedural memory: they may recognize individual cont

Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

arXiv:2608.10932v1 Announce Type: cross Abstract: Understanding camera motion is fundamental to video perception, with applications in spatial intelligence and controllable video generation. Multimoda

UniMod: Enhancing Multi-Modal Medical Diagnosis through Cross-Modality and Within-Modality Alignment

arXiv:2608.10316v1 Announce Type: new Abstract: Multi-modal learning combining medical images and clinical text is promising for disease diagnosis. However, standard multi-modal training leads to shor

safetyarxiv-cs-cv

Agents

VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection

arXiv:2511.19436v2 Announce Type: replace-cross Abstract: Existing Video Detailed Captioning (VDC) methods predominantly rely on costly human annotations or distillation from powerful proprietary mode

agentsarxiv-cs-ai

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: …

model-releasesclem-delangue--x

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B

Where To Look? : Causal Tracing of Vision Encoders in VLM

arXiv:2608.10758v1 Announce Type: new Abstract: Vision-language models can describe an image with remarkable accuracy, yet a more fundamental question remains unanswered: what visual information actua

XCoT-VLA: Executable Chain-of-Thought for Vision-Language-Action Driving

arXiv:2608.10976v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models can connect scene understanding, semantic reasoning, and trajectory generation for autonomous driving. However, verb

safetyarxiv-cs-ai

Local Ai

AquiLLM: An Architecture for Supporting Tacit Knowledge Capture in Research Groups

arXiv:2608.08883v1 Announce Type: new Abstract: Recent advances in retrieval-augmented generation (RAG) and large language models (LLMs) enable researchers to integrate AI into scientific workflows. H

local-aiarxiv-cs-ai

CADEngBench: It Looks Like CAD, but Does It Work? Evaluating Parametric Design, Assembly Reasoning, and Physics Simulation

arXiv:2608.09296v1 Announce Type: new Abstract: A CAD model is not engineering-grade merely because it looks correct. It must satisfy design requirements, respond predictably to parameter changes, sup

Closing the loop in learning with missing data

model-releasesarxiv-cs-lg

arXiv:2608.09030v1 Announce Type: cross Abstract: What should a machine learning model learn when data is missing during training? We look at the learning process from a dynamical systems perspective,

Curriculum Generation under Structured Parametric Environments for Robust Navigation Policies

model-releasesarxiv-cs-lg

arXiv:2608.08545v1 Announce Type: cross Abstract: Robust navigation policies for autonomous agents must generalize across continuously varying environmental conditions such as turn rates, obstacles, f

Decoding Phenotypes: A Framework for Fusing Genomic Language Models and Neuroimaging

arXiv:2608.08926v1 Announce Type: new Abstract: Neuroimaging and genetic testing are two important clinical references for nervous system diseases, offering complementary diagnostic information. Howev

researcharxiv-cs-ai

Applications

Deep probabilistic logic programming for diagnostic reasoning from incomplete information: A case study in stroke detection

arXiv:2608.08561v1 Announce Type: new Abstract: In medical applications, raw data is frequently associated with significant privacy concerns, lending particular importance to the encoding of summary s

applicationsarxiv-cs-ai

Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning

arXiv:2608.09876v1 Announce Type: cross Abstract: Physically consistent motion planning remains a fundamental challenge in embodied AI, as generated trajectories must strictly conform to real-world ex

safetyarxiv-cs-ai

Ethical Framework for Responsible Foundational Models in Medical Imaging

arXiv:2406.11868v2 Announce Type: replace-cross Abstract: The emergence of foundational models represents a paradigm shift in medical imaging, offering extraordinary capabilities in disease detection,

safetyarxiv-cs-ai

FaLCon: Facet-Anchored Retrieval with Late Consensus for Sim2Real Text-Based Person Anomaly Search

model-releasesarxiv-cs-cv

arXiv:2608.09474v1 Announce Type: new Abstract: Text-based person anomaly search requires retrieving real-world pedestrian images from detailed natural-language descriptions using models trained prima

FemWear: A Specialized Wearable Foundation Model for Women's Health

arXiv:2608.08244v1 Announce Type: new Abstract: General wearable foundation models are pretrained across broad sensor streams and populations, but are not designed around women's-health tasks. We intr

Financial Numerical Prediction and Allocation as Token Generation

arXiv:2608.09880v1 Announce Type: new Abstract: Financial prediction typically relies on task-specific regression, ranking, or policy heads, separating the language model from the numerical object ult

safetyarxiv-cs-cv

Tutorials

From Objectives to What Models Learn: A Landau Theory of Invariant Learning

arXiv:2608.09396v1 Announce Type: new Abstract: Invariant learning seeks representations that remain predictive across environments, yet the behavior of its objectives along the regularization path is

tutorialsarxiv-cs-lg

Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

model-releasesarxiv-cs-cl

arXiv:2608.07763v1 Announce Type: new Abstract: Vision-language models (VLMs) have achieved strong performance on tasks such as image captioning, visual question answering, and image-to-text generatio

Tutorials

LAD-COD: Language-Aligned Dense Perception for Camouflaged Object Detection

arXiv:2608.07941v1 Announce Type: new Abstract: Camouflaged object detection (COD) aims to segment objects that exhibit high visual similarity to their surroundings, which reduces foreground-backgroun

tutorialsarxiv-cs-cv

Learning Deep Modality-Shared Self-Expressiveness for Image Clustering with Textual Information

arXiv:2608.08418v1 Announce Type: new Abstract: Leveraging textual information for image clustering has emerged as a promising direction, largely owing to the powerful representations learned by Visio

safetyarxiv-cs-cv

Learning When to See and When to Feel: Adaptive Vision-Torque Fusion for Contact-Aware Manipulation

arXiv:2604.01414v2 Announce Type: replace Abstract: Vision-based policies have achieved a good performance in robotic manipulation due to the accessibility and richness of visual observations. However

safetyarxiv-cs-ro

LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?

arXiv:2503.19990v4 Announce Type: replace Abstract: Many real-world applications of spatial intelligence, such as robotic control, autonomous driving, and automated assembly, require spatial reasoning

Tutorials

Leveraging generative models to assist Monte Carlo sampling

arXiv:2608.07648v1 Announce Type: cross Abstract: Sampling high-dimensional probability distributions is a central task in scientific computing, with applications ranging from Bayesian inference to st

tutorialsarxiv-cs-lg