Read the equation
For each token, select the indices of the four largest group scores. This leaves four groups of thirty-two experts eligible for the next selection. The selection is a discrete ranking operation; ties require the implementation’s top-k tie convention.
Terms & definitions
| Term | Meaning in this source |
|---|---|
| group_idx | Four group indices per token |
How it works
Read the definition from its right-hand expression to the quantity on the left. Its surrounding source records the intended input, state, or parameter context. Where the source leaves an operation unspecified, this page preserves that gap.
Assumptions & limits
Source recovery establishes that the expression appears in this record. Its mathematical validity, implementation behavior, and interpretation must be assessed separately. See the linked source and research notes for the checks available for this version.
Original context
Read the surrounding source passage
-> (T, 8)
Each of the 8 groups is scored by THE SUM OF ITS TOP TWO members.
5. group_idx = topk(group_scores, k=4) -> (T, 4)
Only 4 of 8 groups survive; the other 4 (128 experts) are masked to -inf.
6. topk_idx = topk( masked scores_for_choice , k=8 ) -> (T, 8)
The 8 experts are chosen among the 128 that remain.Exact original notation
group_idx = topk(group_scores, k=4) -> (T, 4)