> Curator / blog
• memory, local-llm, benchmark

The brain got an upgrade. It had to win a bake-off first.

Curator's write-time understanding got stronger this sprint. The new model did not swap in on reputation. It had to beat the model already in place, head to head, on a real store, before it shipped.

The tempting way to upgrade the model inside an AI tool is to read the leaderboard, pick this quarter’s favorite, and swap it in. It usually feels better. Sometimes it is quietly worse, and you never find out, because nobody scored it against the thing it replaced on the data you actually run.

This sprint Curator upgraded the local model that reads your intent when a memory is written. It is a real improvement. What matters is that it did not ship because it was newer or better reviewed. It had to win a bake-off first.

Head to head, on a real store

Every candidate ran against the model already in place, on a frozen copy of a real memory store, scored on the same honest bar we hold all of recall to: does the specific memory an agent needs land where the agent will read it, and does the system refuse to bluff when nothing solid is there. Same data, same grader, one swap at a time.

The replacement shipped only where it beat the incumbent outside the noise. Where the model already in place held its ground, we kept it, and recorded that it won. That sounds obvious and it is rare. It is how you keep an upgrade from being a lateral move wearing a progress badge: a newer model that scores the same or a little worse while everyone assumes forward motion.

Reading your intent

The upgrade earns its keep at write time, in the understanding that decides what a turn actually means and whether it is worth keeping at all. It was measured the same way as everything else, because a gain you cannot prove on your own data is not a gain you should ask a user to run.

Local, and still yours

All of this happens on your machine. The model is local, your store stays encrypted at rest, and the bake-off that chose the upgrade ran on the kind of data you actually keep, not on a public leaderboard that has never seen your projects. An upgrade you can verify against your own memory is worth more than a bigger number on someone else’s benchmark.

Where it stands

Curator is v0.3.0, a working eval-gated pre-release product used every day across our own work, local and reached over MCP. The shared inference path that lets it run without loading its own model is covered in one GPU, every agent, and what the recall is tuned to optimize is in ranked by truth, not recency. It answers to the same proof discipline as BenchProctor and pairs with TheAuditor and Warden. Follow release preparation.

Was this useful?