How to deal with text only vector search across multimodal embedding space? [D]

My data set is a list of images, each equipped with a a couple sentences of text. A user would search primarily with text only. My default approach is using BM25, but how would I facilitate searching with a vector DB and a model that embeds vectors in a multimodal combined space? Here is my dilemma: Do I embed text part and image part as 2 separate individual vectors or do I combine them into 1 vector? If a typical search happens with text only, that would immediately deprioritize all image-only embeddings and only good text matches would float up. This is why I am now considering embedding text and images together but would prefer to hear more opinions on this. Thanks. submitted by /u/AdaObvlada [link] [comments]

Source: r/MachineLearning | 2026-07-28