> Curator / blog
memory, local-llm, benchmark

The brain got an upgrade. It had to win a bake-off first.

Curator's write-time understanding and its recall retrieval both got stronger this sprint. Neither swapped in on reputation. Each replacement had to beat the model already in place, head to head, on a real store, before it shipped.

The tempting way to upgrade the model inside an AI tool is to read the leaderboard, pick this quarter’s favorite, and swap it in. It usually feels better. Sometimes it is quietly worse, and you never find out, because nobody scored it against the thing it replaced on the data you actually run.

This sprint Curator upgraded the local model that reads your intent when a memory is written, and sharpened the retrieval that decides what comes back when you recall. Both are real improvements. What matters is that neither one shipped because it was newer or better reviewed. Each had to win a bake-off first.

Head to head, on a real store

Every candidate ran against the model already in place, on a frozen copy of a real memory store, scored on the same honest bar we hold all of recall to: does the specific memory an agent needs land where the agent will read it, and does the system refuse to bluff when nothing solid is there. Same data, same grader, one swap at a time.

The replacement shipped only where it beat the incumbent outside the noise. Where the model already in place held its ground, we kept it, and recorded that it won. That sounds obvious and it is rare. It is how you keep an upgrade from being a lateral move wearing a progress badge: a newer model that scores the same or a little worse while everyone assumes forward motion.

Reading your intent, ranking your recall

There are two places these upgrades earn their keep. One is at write time, in the understanding that decides what a turn actually means and whether it is worth keeping at all. The other is the recall path, in telling a genuinely strong match apart from a plausible near miss that just happens to read like your question. Both got attention this sprint, and both were measured the same way, because a gain you cannot prove on your own data is not a gain you should ask a user to run.

Local, and still yours

All of this happens on your machine. The model is local, your store stays encrypted at rest, and the bake-off that chose the upgrade ran on the kind of data you actually keep, not on a public leaderboard that has never seen your projects. An upgrade you can verify against your own memory is worth more than a bigger number on someone else’s benchmark.

Where it stands

Curator is v0.3.0, a working eval-gated pre-release product used every day across our own work, local and reached over MCP. The shared inference path that lets it run without loading its own model is covered in one GPU, every agent, and what the recall is tuned to optimize is in ranked by truth, not recency. It answers to the same proof discipline as BenchProctor and pairs with TheAuditor and Warden. Follow release preparation.

Was this useful?