More info about speculative decoding with llama.cpp: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md
Speculative decoding is a technique implemented in llama.cpp that speeds up inference by using a smaller, faster model to predict multiple tokens ahead, which a larger model then verifies in parallel,