Hi, I'm Cort
I'm a backend and distributed systems engineer who likes building things end to end, then taking them apart to understand how they really work. Lately that's meant shipping Cutroom, a music discovery platform with real customers, and writing a from-scratch inference engine to learn how modern serving stacks manage memory. This site is where I write up what I'm building and what I learn along the way.
Read the posts below, or learn more on the About page.
What I'm building
-
Inference engine (from scratch)
A batching layer plus a lifecycle-aware KV cache eviction policy, benchmarked end to end. I'm writing it up here as a series to learn how modern serving stacks manage memory.
-
Cutroom
A music discovery platform connecting artists with curator-led streamers. Live in production with real customers, built end to end: real-time infrastructure on Cloudflare Durable Objects, a Stripe Connect payments stack, and access-tiered HLS streaming.
-
Amazon Inspector
Seven years at AWS, most recently leading vulnerability-scanning systems for EC2 and container images that ran across millions of instances and thousands of accounts.
Featured
-
The three regimes of agentic KV eviction
On real Claude Code traces under memory pressure, LRU does 60 percent more recompute than a strong offline oracle. Three 2026 papers propose policies to close that gap. Measured against the oracle on shared traces, most do not beat LRU; the one signal that reliably helps, lifecycle retirement, does so only in a middle band of pressure and only by a variable amount. Here is the map, with the error bars.
-
Static batching is slow even when your server is idle
At 0.5 requests per second, a nearly idle server, p95 time-to-first-token was 2 seconds under static batching and 45 milliseconds under continuous batching. Same engine, same traffic, identical throughput. The pathology is not a load phenomenon.
-
HuggingFace's greedy decoding isn't always greedy
My inference engine diverged from HF generate() at token 3. Three independent forward passes outvoted the wrapper. The bug was a repetition penalty shipped in the checkpoint's generation_config, applied even under do_sample=False.