Blog
Notes on running models locally.
The hardware, the memory arithmetic, and the handful of numbers that decide what your machine can actually run — written out properly, so ModelFit can point at them instead of repeating itself.
-
LLM runtimes and inference
What inference actually is, what an engine does, and where the runtime begins — the layers of software between a model file and a token on your screen. With the tools you have heard of, Ollama and llama.cpp and vLLM, placed into those layers.
Read the post → -
Local LLM fundamentals
Parameters, quantization, KV cache, memory bandwidth and tokens per second. What each number means, how they interact, and why the answer to “can my machine run this?” is nearly always about memory rather than the GPU you were told to worry about.
Read the post →