A lot of tools wrap an inference engine like llama.cpp to let you run LLMs locally. Few of them actually benchmark on your hardware to tell you which models you can realistically run.

Months ago I found myself with tens of gigabytes of models sitting on my hard drive, so I decided to figure out which configuration was actually the fastest - at reading prompts and generating tokens. I wrote a PowerShell script to measure exactly that.

It quickly grew into something more structured, once I realized it could help other people figure out what their hardware can actually do. That became calibr: a benchmark and recommender tool that downloads models from Hugging Face and runs them locally under different configurations. When it's done, you get stats on every model and configuration it tried: prompt and generation speed, VRAM and RAM usage, VRAM-to-RAM spillover, and load time.

In this talk I'll walk through what calibr is, where it's heading, and how it works under the hood - along with some of the challenges I ran into while building it.