RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving
DGX agentarXiv:2606.22840v1 Announce Type: new Abstract: We present RLM-Cascade, a proxy-layer system that applies speculative decoding at the response level to reduce LLM API costs without requiring model arc