Knowledge catalogue
Ethan Mollick invited readers to discuss the distinctions between **pure large language models (LLMs)**, **multimodal LLMs** that process multiple data types (e.g., text, images), and **multimodal LLM
Brace yourselves. We just entered a new era of frontier multimodal models. DeepSeek-V4-Flash-Vision-Exp advances multimodal agent performance. Also, pay attention to Ox Alpha 1M token contexts with te
DeepSeek-V4-Flash-Vision-Exp is now live on the DeepSeek API Platform! 🚀 🔹 This experimental multimodal model matches DeepSeek-V4-Flash on text capabilities—including agents, reasoning, and world know
Multimodal API support 🔌 🔹 Set model='deepseek-v4-flash-vision-exp' 🔹 Images are tokenized for billing: up to 384 tokens each, at V4-Flash pricing 🔹 Supports Chat Completions, Messages & Responses 🔹 S
👏👏 Introducing Qwen3.7-Plus — a multimodal agent model that unifies vision and language into one versatile agent foundation. ✅ Multimodal interactive hybrid agent: unified GUI & CLI operation across v
Demo2 showcases Qwen's multimodal interactive hybrid agent capabilities, likely demonstrating the integration of multiple data types (text, image, audio, video) with interactive features and hybrid pr
In May, we integrated 11 new models spanning image, 3D, audio, video, and multimodal. The highlights: → Krea 2 — style-first image generation, live as a Partner Node on day one. Competes on how the fr
VLM Performance：Qwen3.6 is natively multimodal, and Qwen3.6-35B-A3B showcases perception and multimodal reasoning capabilities that far exceed what its size would suggest, with only around 3 billion a
Building search across text, images, audio, and video? Multimodal search uses vector embeddings to retrieve content based on meaning, not just keywords or metadata. Less manual tagging. More flexible
1M-token, multimodal decision models are here! But there is more to this one. You can classify, extract, and summarize with 1M tokens of text and image input. ScaleDown's Constrained Decoding Models r
What's better than an open-weight multimodal model release? Well, the technical report. I just love how these labs like Qwen and DeepSeek continue to drop gem after gem. Qwen3.8-Flash is the latest in
Interested in designing and building agent harnesses for long-horizon, multimodal tasks? Nomic is hiring a Harness Engineer. We're post-training models and an agent harness that can perform most tasks
[1/7] 🚨 We're releasing LAION-BVD: a 10-million-hour open video dataset for multimodal pre-training. - 1.3B video URLs from CommonCrawl - 80M downloaded videos - 10M video hours - 55M captioned clips
The mystery model Ox Alpha is now available in @AutoClawAIer by @Zai_org It's official name is GLM-5.3-Flash, a next-generation multimodal model for vision-language understanding, code generation, and
Introducing GLM-5.3-Flash - Leading capabilities at a highly competitive price - Natively multimodal with a 1M-token context window - A 320B-A18B model released under the MIT License - Previously prev
⚡Meet Qwen3.8-Flash, a multimodal MoE and an early preview of the Qwen4 architecture, now open-weight! The production version Qwen3.8-Flash will be available soon via QwenCloud API at just 0.16/1M inp
👀 Seeing is just the beginning. With Qwen-MM-Plugins, turn your favorite agent harness multimodal-native — read images, videos & documents, edit videos, work with 3D/CAD, and more. From multimodal mod
Meta released Muse Glimmer 30B: multimodal model for your Claw/Pi setups 🔥 we tested and fine-tuned the model for you, and shipped day-0 support in transformers and llama.cpp, including DFlash for 2-4
Bonsai 27B is a big step for local AI: 27B-class multimodal capability in a phone-class footprint. Congrats to @PrismML on the launch. Try Ternary Bonsai 27B on Together AI: https://www.together.ai/mo
Huge if true! We are talking about a 27B multimodal model that runs locally on a phone. That's wild! Bonsai 27B reaches up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090.
Today, we’re announcing Bonsai 27B: the first 27B-class model to run on a phone. Bonsai 27B is the new multimodal flagship of the Bonsai family. Based on Qwen3.6 27B, it brings a new capability tier t
Multimodal prompting is clearly the future. I love experimenting with new ways to interact with agents. As a researcher and engineer, I've found that the richer the inputs to the agent and the richer
GLM-5V-Turbo Tech Report: Toward a Native Foundation Model for Multimodal Agents This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcemen
.@GoogleDeepMind EmbeddingGemma 2 is now available on Ollama. Made for consumer devices and is multimodal. ollama pull embeddinggemma-2 Meet EmbeddingGemma 2, our first natively multimodal open model
Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workl
pplx-decider-v1.1-27b, our updated open weights multimodal decision model, is now available. It scores the highest on the new @huggingface Decision Index 0.3 benchmark. pplx-decider-v1.1-27b costs hal
FLUX 3 is now live on Together AI. @bfl_ai’s new multimodal model generates video and synchronized audio together, with up to 20-second clips, multiple shots, and control from text, images, or keyfram
Inkling-Small is now live on Together AI. @thinkymachines’ new open-weight multimodal model delivers similar performance to Inkling at one-quarter the size, built for coding, agents, and general multi
Excited for our first general model Inkling -- open weights, 975B, natively multimodal (text, image, audio). Available on Tinker, HuggingFace and partners. It is yours to personalize and use openly. I
Never too late for any company to get in on open models. Inkling is a MoE transformer with 975B (41B active). Native multimodal support. Context lengths: 64K and 256K. Will be testing and providing th
This is for pretraining based on nemotron nvp4 figures, mfu etc Large context, multimodal, long RL we are seeing now could make it a multiple of this Worth noting how good the lite model is, similar t
Ethan Mollick notes that image input represents the primary advanced capability of current AI models, and that tool‑use can effectively replace outputs from non‑omni models. He observes that multimoda
This post discusses the evolution of multimodal prompting in AI agent interactions, highlighting how users can now communicate with AI systems using multiple input types beyond text. The author provid
Multimodal prompting enhances coding agents by enabling them to process and integrate multiple types of input data—such as text, images, and code snippets—simultaneously, improving their ability to un
Amazing deep dive from the @togethercompute team on serving MiniMax M3 in production. M3 with its 1M context, native multimodality and MiniMax Sparse Attention requires real work across paged decode,
MiniMax-M3 combines 1M context, native multimodality, and MiniMax Sparse Attention. The next layer is serving it efficiently: KV-block-major sparse attention, paged MSA decode, optimized index scoring
VLM Performance：Qwen3.6-27B is natively multimodal, supporting both vision-language thinking and non-thinking modes in a single unified checkpoint — the same as Qwen3.6-35B-A3B. It handles images and
Google's Gemma 4 31B is a dense multimodal model from Google DeepMind now available on Together AI's serverless infrastructure via the endpoint `google/gemma-4-31B-it`. It features a 256K context ...
Introducing the largest open-source multilingual, multimodal RL environments 🌍 📄 OCR: 1M+ document pages in 22 languages 🎙️ ASR: speech in 102 languages 12M+ tasks across two OpenEnv Environments, ope
Introducing Decision 3.0: towards open multimodal foundation decision models ⚡ Open and state-of-the-art from 0.6B to 27B: #1 on both Jev Decision Index boards in our evaluation, on a new Pareto front
deep agents just shipped a round of multimodal upgrades! 1. broader file support on OpenAI Responses: docx, pptx, xlsx, csv, json, markdown, not just pdfs 2. opt-in binary offloading to keep big files
This is both fun and a useful measure of how much better multimodal AI has gotten In a short time. Can AI tell if you've built your IKEA furniture wrong? Our new benchmark, the Furniture Assembly Benc
you can now build @LangChain agents w/ agentic video understanding! google is consistently at the forefront of multimodal processing; this is the video analog of 'agentic vision' token usage + cost is
Qwen 125B/6B models with a 1‑million‑token context window and multimodal support have been released on Alibaba’s OpenCode Go platform. The new Qwen3.8‑Flash variant is also now available in the same e
Gemini Omni 1.1 Flash is live in ComfyUI via Partner Nodes. Google's multimodal video model, one node, five task types: → text-to-video → image-to-video → reference-to-video → edit → extend Up to 4K.
Perplexity AI has released its new language model GLM 5.3 on Perplexity Computer. Designed for long‑context, multimodal agent workloads, GLM 5.3 outperformed the previous GLM 5.2 on the company’s WAND
Meet Gemini Omni 1.1 Flash ⚡️ Our newest multimodal model for video generation and editing. It now features your favorite creative controls from Veo, plus brand new capabilities. Enjoy features like 4
Me, 2023, 2024, 2025…. Multimodal LLMs are going to have problems with maps. 2026: It’s ok, you live in Venecky. Not far Tounesseo. @GaryMarcus It keeps getting crazier “Mom, did you know that we live
We promised open weights for Qwen3.8. Now, time to meet them! 🎉 ⚡ Qwen3.8-27B: - A native multimodal dense model. With just 27B parameters, it outperforms Qwen3.7-Plus overall and shines in real-world
Glad to see Google sharing data on how Gemini is being used. Especially interesting is that the usefulness of multimodal AI for manual labor may be greater than expected. https://blog.google/innovatio
We stress tested many frontier AI models for multimodal medical reasoning (including GPT-5, Claude 3.5, Gemini 2.5 Pro). They’re not ready. Faulty reasoning, use of inappropriate shortcuts, hallucinat
Anthropic's Claude 3.5 Sonnet (M3) model features sparse attention mechanisms, 1 million token context length, and multimodal capabilities, with Together AI optimizing the serving infrastructure to en
Thinking Machines is impressive. In a couple hours I just fine tuned my own Qwen3.5-397B model this afternoon. Fast usable multimodal is also going to enable very mind-blowing personal AI. People talk
I think people don't realize why Gemini Omni is different than other video AIs. It is fully multimodal, so it can edit video natively, too I took the famous 'train ' movie from 1896 & made it a bullet
streaming from modern agents is pretty complex! especially with - parallel tools / subagents - multimodal content - human in the loop events our new streaming primitives make all of this ergonomic! Ag
Gemini Omni is an extension of Google's Gemini model that enhances its multimodal capabilities by enabling seamless integration of text, audio, image, and video inputs and outputs. This advancement al
Introducing Starchild-1 from @odysseyml, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear.
Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID
Introducing Qwen3.6-Plus from @Alibaba_Qwen, a 1M-context model built for real-world agents, agentic coding, and multimodal reasoning. AI natives can now use Qwen3.6-Plus on Together AI and benefit fr
@NVIDIA Nemotron 3 Nano Omni is now on Together AI. Enterprise multimodal AI — video, audio, image, documents & text — optimized for speed and scale. ✅ ~3B active params, 9x higher throughput ✅ Fully