OpenPM: Auditable Point-in-Time Evaluation for LLM Portfolio-Management Agents

arXiv:2608.09988v1 Announce Type: cross Abstract: Large language models are increasingly used to read markets, assess risk, and allocate capital. However, reported results for LLM trading agents can be inflated by look-ahead leakage, optimistic execution, and risk mandates that are described but not enforced. We present OpenPM, an auditable point-in-time evaluation framework for LLM portfolio-management agents. In OpenPM, an agent manages a $1M long-only book over the S&P 500 universe using market data at five-minute intervals. Every record visible to the agent must be available at the decision time. Natural-language risk mandates are converted into typed constraints and enforced on the executed portfolio. Each run produces audit artifacts, including a contamination certificate, a cost-sensitivity curve, and a constraint-adherence report. We also build a reference agent named the tiered allocator, where typed analysts score candidates, a constructor LLM proposes weights, and a deterministic critic guarantees feasibility. We isolate constructor behavior by capturing analyst evidence once and replaying it across constructor models. In our short-window case study, stronger constructors show modest and model-dependent gains over equal weighting on the same pool, but analyst quality matters more than constructor choice, and turnover is the main cost driver. All returns are upper bounds on a single frozen window without market impact, not validated alpha.

Related

• SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection
• OTAP: Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories
• Bi-Predictability: A Real-Time Signal for Monitoring LLM Interaction Integrity

Source: arXiv cs.CL | 2026-08-12