KV Cache

ذاكرة المفاتيح والقيم

Definition

Storing the attention keys and values of previous tokens to avoid recomputing them when generating each new token.

Definition

A KV cache stores attention keys and values of previously processed tokens so each new token needs only incremental computation during autoregressive generation.

Learn and review

Suggested terms to learn next