MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization

arXiv:2604.12237v1 Announce Type: cross Abstract: In drug discovery, molecular optimization aims to iteratively refine a lead compound to improve molecular properties while preserving structural similarity to the original molecule. However, each oracle evaluation is expensive, making sample efficiency a key challenge for existing methods under a limited oracle budget. Trial-and-error approaches require many oracle calls, while methods that leverage external knowledge tend to reuse familiar templates and struggle on challenging objectives. A key missing piece is long-term memory that can ground decisions and provide reusable insights for future optimizations. To address this, we present MolMem (extbf{Mol}ecular optimization with extbf{Mem}ory), a multi-turn agentic reinforcement learning (RL) framework with a dual-memory system. Specifically, MolMem uses Static Exemplar Memory to retrieve relevant exemplars for cold-start grounding, and Evolving Skill Memory to distill successful trajectories into reusable strategies. Built on this memory-augmented formulation, we train the policy with dense step-wise rewards, turning costly rollouts into long-term knowledge that improves future optimization. Extensive experiments show that MolMem achieves 90% success on single-property tasks (1.5imes over the best baseline) and 52% on multi-property tasks using only 500 oracle calls. Our code is available at https://github.com/REAL-Lab-NU/MolMem.

Related

• Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models
• CROP: Conservative Reward for Model-based Offline Policy Optimization
• Curriculum-based Sample Efficient Reinforcement Learning for Robust Stabilization of a Quadrotor
• The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping
• SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents
• Self-Organizing Dual-Buffer Adaptive Clustering Experience Replay (SODACER) for Safe Reinforcement Learning in Optimal Control

Source: arXiv cs.AI | 2026-04-15