Speculative Decoding
فك الترميز التخميني
Definition
Speeding up generation with a small model that proposes several tokens and a large model that verifies them in one pass.
Definition
Speculative decoding speeds up generation by letting a small draft model propose several tokens that a large model verifies in one parallel pass, preserving the large model's output distribution.
