Tag

#Mixture Of Experts

Every The Latent Daily story, tool, model and resource tagged “Mixture Of Experts”.

2 items

Paper of the day: Switch Transformers and the rise of mixture-of-experts

Switch Transformers showed that routing each token to one expert lets models grow to a trillion parameters without proportional compute. How MoE works and why it matters.

AI DeskOct 184 min read

Models tagged #Mixture Of Experts

DeepSeekJan 20, 2025

DeepSeek-R1

DeepSeek's open-weight reasoning model trained with large-scale reinforcement learning, matching leading reasoning models at release under MIT.

Type: Reasoning
Context: 128K
Weights: Open