Computation and Data Movement for Inference

Media reports SemiAnalysis Source language: English

Excerpt

Mixture of Experts, now widely used in frontier models, has changed both the structure of serving and the economics of useful inference. It did more than increase parameter count.

Short excerpt from the source — open the original for the full text.

Sources

← Back to Daily AI Report