Start Classifying: Categorical Critics for LLM Reinforcement Learning
DGX agentarXiv:2608.02181v1 Announce Type: new Abstract: Proximal Policy Optimization (PPO) for large language models typically trains its critic by mean-squared-error (MSE) regression on scalar value targets.