[P] PCA before truncation makes non-Matryoshka embeddings compressible: results on BGE-M3 [P]
Applying PCA as a rotation step before naively truncating embeddings from non-Matryoshka models like BGE-M3 can recover much of the retrieval quality that is otherwise lost when simply chopping dim...