mixture of experts
7 published articles
Grok 4.5
Cursor's Grok 4.5 was built by AI agents, not humans. That's the real story.
Cursor's Grok 4.5 is a Mixture-of-Experts model built using reinforcement learning in environments created by earlier AI agents, not humans. It handles complex, long-duration tasks across software engineering, data science, finance, and law, and it's available now.
2026-07-10
Portfolio strategy
Alibaba's Qwen is building a model for every AI job, not just one to rule them all
Alibaba Cloud's Qwen team quietly released a 35B MoE agent world model, three new ASR models, and an image-generation RL report, revealing a strategic bet on breadth over spectacle in the AI model race.
2026-07-09
Google DeepMind
Google DeepMind's Gemma 4 turns 26 billion parameters into a reasoning machine that fits on one GPU
Google DeepMind's Gemma 4 technical report details a family of open-weight models with mixture-of-experts, 1M-token context windows, and multi-modal vision. The release signals a strategic play to bring frontier-level reasoning to developers without the cost of proprietary APIs.
2026-07-09
Artificial intelligence
Aleph alpha's new megakernel library cuts moe inference latency by 200%
Alpha-MoE fuses multiple operations into a single persistent kernel to achieve up to 200% inference speed gains over Triton-based kernels in vLLM and SGLang, targeting FP8-precision MoE models.
2026-07-09
Artificial Intelligence
Ai2's EMO makes modular AI emerge from data, not human rules
Ai2's new MoE model, EMO, uses a novel training method that lets expert modules emerge naturally from data, enabling selective expert use with minimal performance loss. The model matches standard MoE performance on benchmarks while offering vastly improved modularity.
2026-07-07
Deep Dive
Aleph Alpha builds theoretical inference model for DeepSeek: Deriving performance from hardware primitives
Aleph Alpha created a theoretical inference model for DeepSeek v3 to estimate throughput from hardware parameters, analyzing trade-offs across GPU setups to help practitioners optimize performance and cost for large MoE models.
2026-07-05
LLM Inference Optimization
Aleph Alpha builds a theoretical inference model to decode DeepSeek V3 performance from hardware primitives
Aleph Alpha's theoretical model predicts DeepSeek V3 inference performance from hardware parameters alone, revealing how GPU count and interconnect bandwidth shift the bottleneck between compute, memory, and communication.
2026-07-04