Mixture of Experts

خليط الخبراء

Abbreviation: MoE

Definition

An architecture containing several “expert” networks and a router that selects only a few of them for each input.

Definition

A mixture-of-experts architecture contains many expert sub-networks and a router that activates only a few per input, scaling parameters without proportionally increasing compute.

Learn and review

Suggested terms to learn next