KV Cache
ذاكرة المفاتيح والقيم
Definition
Storing the attention keys and values of previous tokens to avoid recomputing them when generating each new token.
Definition
A KV cache stores attention keys and values of previously processed tokens so each new token needs only incremental computation during autoregressive generation.
