Mixture of Experts
خليط الخبراء
Definition
An architecture containing several “expert” networks and a router that selects only a few of them for each input.
Definition
A mixture-of-experts architecture contains many expert sub-networks and a router that activates only a few per input, scaling parameters without proportionally increasing compute.
