Research notes · Paper reading · Engineering
Longer notes on GPU systems, compilers, distributed systems, and the ideas encountered along the way.
A persistent kernel that keeps DSpark's vocabulary-wide Markov matrix on chip: 8–12× faster draft walks on H100 and up to +12% per decoding round with identical greedy outputs, plus bf16 tensor-parallel, B200 and other serial-head versions.
What guessing a whole block blind really costs in speculative decoding, why one previous token wins almost all of it back, and how a better head and 2× faster training follow from that.
How an 81-kernel CUDA Graph uses PDL to recover 97.4% of a persistent megakernel's end-to-end throughput on H100.