Momba: Network Modernization Improves Multi-Objective Reinforcement Learning

arXiv:2608.07180v1 Announce Type: cross Abstract: Recent advances in deep reinforcement learning (RL) have shown that improving neural network architectures can yield substantial gains in sample efficiency and asymptotic performance without altering the underlying algorithms. In contrast, work on multi-objective reinforcement learning (MORL), which aims to discover a set of policies that balance trade-offs among conflicting objectives, has predominantly focused on algorithmic innovations, leaving the area of architectures underexplored. While the optimal policies and value functions can differ significantly depending on the trade-offs, MORL algorithms commonly represent them with simple feedforward networks conditioned on the trade-off. This raises the question of whether the performance of the algorithms could be improved with more expressive function approximators. In this paper, we integrate recent advances in neural network design: (i) observation and feature normalization, (ii) weight normalization, and (iii) modeling of distributional returns with an entropy-regularized MORL algorithm. The empirical results across standard continuous control benchmarks demonstrate that these changes substantially improve the quality of the produced solution sets without requiring major changes to the underlying algorithm.

Related

• Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
• ProDVI: Programmatic Dynamics Priors for Value Network Initialization
• From Ticks to Flows: Dynamics of Neural Reinforcement Learning in Continuous Environments
• Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms

Source: arXiv cs.AI | 2026-08-10