From Raw Features to Effective Embeddings: A Three-Stage Approach for Multimodal Recipe Recommendation

arXiv:2511.19176v3 Announce Type: replace Abstract: Recipe recommendation has become an essential task in web-based food platforms. A central challenge is effectively leveraging rich multimodal features beyond user-recipe interactions. Our analysis shows that even simple uses of multimodal signals yield competitive performance, suggesting that systematic enhancement of these signals is highly promising. We propose TESMR, a 3-stage framework for recipe recommendation that progressively refines raw multimodal features into effective embeddings through: (1) content-based enhancement using foundation models with multimodal comprehension, (2) relation-based enhancement via message propagation over user-recipe interactions, and (3) learning-based enhancement through contrastive learning with learnable embeddings. Experiments on two real-world datasets show that TESMR outperforms existing methods, achieving 7-15% higher Recall@10.

Related

• SIGMA: A Semantic-Grounded Instruction-Driven Generative Multi-Task Recommender at AliExpress
• Improving Large-Scale Recommender Systems with Auxiliary Learning
• LiveGraph: Active-Structure Neural Re-ranking for Exercise Recommendation

Source: arXiv cs.LG | 2026-04-23