ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieval, structured timelines, LLM summaries, agentic memory---preserve the longitudinal signal clinical reasoning needs has not been measured. We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy (T1--T9), and L0--L4 deterministic + L5 human-audit validation (98.92% agreement). We evaluate eight history representation strategies---a no-context floor, extit{last-visit-only}, extit{full-context}, BGE-M3 extit{dense-retrieval}, two compression schemes, and two agentic-memory systems (extit{Mem0}, extit{A-Mem})---across four backbones (DeepSeek-V3, GPT-4o-mini, Haiku4.5, Sonnet4.6) on 6{,}271 questions: 32 cells, 200{,}672 predictions. Four findings: (SP4) a controlled T3 injection probe isolates compression-induced extit{relation} loss---with the attribution sentence present extit{before} construction, extit{Mem0}, extit{A-Mem} and extit{llm-summary} still recover only 0--5.3% of the injected positives; (SP1) compressed strategies pay an aggregation tax on multi-visit trends and cross-patient comparisons; (SP2) the blind-to-full gap spans +29.8pp (GPT-4o-mini) to +62.7pp (Haiku); (SP3) abstention scales non-monotonically with context length. On the Pareto frontier Haiku dominates Sonnet under extit{full-context} (25.76 vs. 106.21), inverting the ``biggest backbone wins'' heuristic.

Related

• A Source-Grounded Framework for Constructing and Evaluating Progressive Multimodal Diagnostic Dialogues from Clinical Case Reports
• ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV
• RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care
• TSVer: A Benchmark for Fact Verification Against Time-Series Evidence

Source: arXiv cs.CL | 2026-09-02