Multi-Head Attention

الانتباه متعدد الرؤوس

Abbreviation: MHA

Definition

Running several attention mechanisms in parallel with different projections and then combining their results.

Definition

Multi-head attention runs several attention functions in parallel on different learned projections and concatenates their outputs.

Learn and review

Suggested terms to learn next