Agents
The root cause: request-level engines never see that a series of LLM calls belongs to one longer workflow. ThunderAgent adds that missing vi…
The root cause: request-level engines never see that a series of LLM calls belongs to one longer workflow. ThunderAgent adds that missing view. It treats each agent workflow as a schedulable program,
The root cause: request-level engines never see that a series of LLM calls belongs to one longer workflow. ThunderAgent adds that missing view. It treats each agent workflow as a schedulable program, tracking its phase, KV footprint, and node placement.
Related
- ThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale
- 2/ ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System Paper: https://arxiv.org/abs/2602.13692
- Agentic inference wastes GPUs on KV cache thrashing. ThunderAgent fixes it at the scheduler level: 2.5x higher single-node throughput and ~1…
Source: Together AI (X) | 2026-07-29