Speculative Decoding

فك الترميز التخميني

Definition

Speeding up generation with a small model that proposes several tokens and a large model that verifies them in one pass.

Definition

Speculative decoding speeds up generation by letting a small draft model propose several tokens that a large model verifies in one parallel pass, preserving the large model's output distribution.

Learn and review

Suggested terms to learn next