Purpose & architecture
SwitchTransformer is local Generator Core registry configuration 99, created by NewSwitchTransformer in the mixture package. Its exact constructor and configuration settings are preserved below.
The shared implementation initializes a router and expert arrays, selects top-k experts and combines their outputs with routing probabilities. Train returns output mean squared error plus a penalty encouraging the selected probabilities toward a uniform distribution.
An expert count and routing configuration are concrete local settings; they are not evidence of independently trained expert models. The inspected Train method only computes losses. Model labels such as SwitchTransformer and GLaM do not supply their official architecture or weights.
This source generation
This record follows the claude/improve-math-logic-ehIW1 branch of Janady13/complete-agi-core at commit 33aa84f2aa61cedbe010240ada7f4efbdfe795a7. The snapshot identifies the source inspected in this audit. It is not a statement about which revision is currently running on a device.
| Field | Source record |
|---|---|
| Revision | 33aa84f2aa61cedbe010240ada7f4efbdfe795a7 |
| Branch | claude/improve-math-logic-ehIW1 |
| Recorded generation | Generator Core · mixture |
Recovered implementation structure
These paths locate the components and entry points represented by this build. The source register contains their inspected revisions.
- generator_core/pkg/generators/mixture/mixture.go
Behavior & verification limits
An expert count and routing configuration are concrete local settings; they are not evidence of independently trained expert models. The inspected Train method only computes losses. Model labels such as SwitchTransformer and GLaM do not supply their official architecture or weights.
Original project documentation
The following documentation is preserved as a dated source account. Claims in it retain the source’s wording and must be read alongside the implementation notes above.
func NewSwitchTransformer() *MixtureGenerator {
g := NewMixtureGenerator(99, "SwitchTransformer", 64, 1)
g.Config().Params["capacity_factor"] = 1.25
g.Config().Params["switch"] = true
return g
}Equations in this build
1 linked mathematical records. Each opens its notation, interpretation and source.