Deploying Open-Source LLMs Privately

Self-hosting is usually justified on privacy grounds, and the argument is weaker than assumed: it changes who owns the controls, not whether controls are needed. This cursus examines the decision honestly, the reasons that hold up better than privacy, and the middle options the either-or framing hides. Then the serving stack: why generation is memory-bandwidth-bound, how continuous batching and KV cache management decide throughput, and sizing hardware. Then operating it, including the safety layer you inherited and must now rebuild.