mixture of experts
3 published articles
Artificial Intelligence
Ai2's EMO makes modular AI emerge from data, not human rules
Ai2's new MoE model, EMO, uses a novel training method that lets expert modules emerge naturally from data, enabling selective expert use with minimal performance loss. The model matches standard MoE performance on benchmarks while offering vastly improved modularity.
2026-07-07
Deep Dive
Aleph Alpha builds theoretical inference model for DeepSeek: Deriving performance from hardware primitives
Aleph Alpha created a theoretical inference model for DeepSeek v3 to estimate throughput from hardware parameters, analyzing trade-offs across GPU setups to help practitioners optimize performance and cost for large MoE models.
2026-07-05
LLM Inference Optimization
Aleph Alpha builds a theoretical inference model to decode DeepSeek V3 performance from hardware primitives
Aleph Alpha's theoretical model predicts DeepSeek V3 inference performance from hardware parameters alone, revealing how GPU count and interconnect bandwidth shift the bottleneck between compute, memory, and communication.
2026-07-04