What if Chain-of-Thought wasn’t lossy? Exploring reversible logic (Toffoli/Fredkin-style) for edge LLMs

Right now standard CoT is a one-way street. You generate forward, dump a pile of scratchpad tokens into the KV cache, and pray the model doesn’t hallucinate halfway through. On phones/laptops that creates two ugly problems: Errors compound. If each step is only (1-ε) reliable, an N-step chain dies as ~e^{-Nε}. There’s no cheap way to check intermediate steps without another full forward pass. KV cache just grows. Intermediate tokens sit there forever. Blind pruning breaks context, so memory is O(N) with reasoning length. The idea: What if every reasoning step was an approximately reversible transformation in embedding space — basically soft analogues of classical reversible gates (Toffoli, Fredkin)? If the step is bijective (or close enough): • Instant sanity check via cycle consistency: run f⁻¹(f(X)) and see if you get X back. Big deviation → the step leaked information / hallucinated. No extra judge model needed • Bennett-style uncomputation: once you reach the answer you can systematically “uncompute” the intermediate scratchpad tokens and free the KV slots. In theory this can drop active working memory from O(N) toward O(log N) (classic reversible computing result). • Cheap backtracking: hit a dead end? Apply the inverse in-place instead of nuking the whole cache or re-encoding the parent state. I’m looking at this as a lightweight reasoning harness / external scratchpad for models we actually run locally, not by trying to make vanilla dense attention layers run backwards. Softmax + the usual projection matrices are lossy; the reversibility would live in structured latent registers, invertible coupling layers, or RevNet-style adapters sitting on top of the frozen model. An inverse pass on a low-rank adapter is still a forward pass, but it’s orders of magnitude cheaper than spinning up a second big judge model or doing multi-path rejection sampling. Open questions / sanity checks wanted • Has anyone already played with invertible projection layers or reversible residual blocks specifically for intermediate CoT tokens? • How to handle non-invertible attention heads? (e.g., using lightweight invertible coupling layers / RevNets as an external reasoning scratchpad layer vs. trying to touch native weights) • Would you rather see this first as a structured decoding harness (Outlines/Guidance-style registers that stay invertible) or as a small fine-tune that teaches the model cycle-consistent lemma steps? • Any obvious reasons this is doomed on real transformers / real KV caches? submitted by /u/Fear_ltself [link] [comments]

Source: r/LocalLLaMA | 2026-08-22

Related