Notes
Working notes on expert routing, model compression and speech systems. Where a number appears it comes from a run I can point at, and the command that produced it is included.
What a voice says before the words do
A speech system can transcribe perfectly and still handle a call badly, because the words were fine and the caller was furious. Notes on why affect belongs in the pipeline, and why a confidence score is not an explanation.
Profiling a MoE model without loading it
A 48-layer, 512-expert checkpoint profiled in 4.98 seconds on CPU. The trick is that expert routing is decided by a few megabytes of router weights, not by the hundreds of gigabytes behind them.
Expert routing is flatter than you think
Profiling all 48 layers of Qwen3-Coder-Next gives a routing Gini of 0.1006. Almost nothing is dead, almost nothing dominates — which changes what pruning a Mixture-of-Experts checkpoint can and cannot buy you.