I built a new attention mechanism (wave field) — runs 128K context where standard attention OOMs, 80+ tok/s on laptop CPU

Hey r/LocalLLaMA — solo researcher here. I built a new attention architecture and want independent testers. Wave Field LLM replaces O(N²) dot-product attention with FFT wave convolution on a field. Training is O(N log N). Inference is O(1) per token — constant speed and memory even as context grows. Important: this is a base completion model, not a chat model. Trained from scratch. No RLHF, no safety tuning, no instruction fine-tune. What I've measured so far: 80+ tok/s on Mac laptop CPU (no GPU for inference) 128K context runs where standard attention OOMs Models from 130M to 1.5B params 130M zero-shot (DCLM CORE) vs GPT-2 124M: Wave Field avg: 46.8% | GPT-2: 26.5% PIQA: 61.7% vs 50.0% ARC Easy: 43.8% vs 25.0% At 32K on H100: 21.8x faster, 5.3x less memory than standard attention. Links: https://github.com/badaramoni/wave-field-llm https://wavefieldlab.com/ Demo: https://www.youtube.com/watch?v=zH7ICaY5iz4 Looking for honest feedback: Benchmark methodology — sound or not? How does quality compare to other ~130M base models? Anything that looks off ? it produces harmful answers Active research, arch v9, patent pending. Happy to answer in comments. If you test it, reply with hardware + tok/s + a sample completion. Bad results welcome. submitted by /u/Murky-Sign37 [link] [comments]

Related

• Qwen Introduced FlashQLA
• Meta-Attention: Bayesian Per-Token Routing for Efficient Transformer Inference
• MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training

Source: r/LocalLLaMA | 2026-07-15