Research · The Decoder ·
Researchers train AI model that hits near-full performance with just 12.5 percent of its experts
Researchers at the Allen Institute for AI and UC Berkeley built EMO, a mixture-of-experts model whose experts specialize in content domains rather than word types. The team says removing three-quarters of the experts reduced performance by only about one percentage point, suggesting MoE models may be more practical on