Equal-information evaluation of credit assignment in modular financial reinforcement learning

arXiv:2610.14750v1 Announce Type: new Abstract: Credit assignment in modular financial reinforcement learning is difficult to evaluate because an attributed signal may improve a component loss without adding decision-relevant information or improving economic outcomes. We propose an evaluation framework combining equal-information reconstruction controls, fixed-utility comparisons and prespecified downstream guardrails. The framework is instantiated in an attribution-guided modular trading architecture and tested in three controlled studies covering update allocation, supervision-target representation and portfolio risk control. We find that residual-based allocation is exactly reproduced by an ordinary loss-budget control under the disclosed interface; direct shortfall supervision improves component prediction but fails decoded-outcome preservation; and component-wise critic supervision worsens risk-condition certainty equivalent and drawdown relative to scalar supervision under fixed utility. These results identify three distinct evaluation failures: redundant attribution signals, representation-induced prediction trade-offs, and misalignment between component supervision and downstream risk outcomes. A preserved historical engineering case further shows why module-level learning claims require run-level provenance. The contribution is a reusable methodology for evaluating credit assignment in expert systems operating under partial observability, delayed feedback and economic risk constraints.

Related

• Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning
• Principled Analysis of Deep Reinforcement Learning Evaluation and Design Paradigms
• When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning

Source: arXiv cs.AI | 2026-10-15