Multimodal Embedding & Reranker Models with Sentence Transformers

The Sentence Transformers v5.4 update introduces first-class multimodal support, enabling the same familiar API to encode and compare texts, images, audio, and videos using both SentenceTransformer (embedding) and CrossEncoder (reranker) model classes. Multimodal embedding models map inputs from different modalities into a shared embedding space, while multimodal reranker models score the relevance of mixed-modality pairs — such as a text query against image or combined text-image documents. This enables cross-modal retrieval pipelines, including two-stage retrieve-and-rerank workflows, with support for models such as Qwen3-VL-Embedding and Qwen3-VL-Reranker.

Related

• Safetensors is Joining the PyTorch Foundation
• Build knowledge agents without embeddings
• Opus 4.6 Fast Mode available on AI Gateway
• Zero Data Retention on AI Gateway

Source: tools