A memory you can measure beats one you just trust
Most AI memory asks you to trust that recall is good. Curator benchmarks its own memory on a frozen copy of a real store, with the test and the result that would prove it wrong both written down before the run. Quality you can measure, not quality you assert.
Every memory tool tells you its recall is good. Almost none can show you. You are asked to take it on faith that the right thing comes back when you ask, and on the day it doesn’t, you find out the expensive way: in a live session, from an agent that built confidently on something stale.
Curator treats its own memory the way you would treat a search engine you were about to depend on. It gets benchmarked, on purpose, over and over, and nothing ships to your session until the numbers say it earned the spot.
The claim has to survive a test written before the run
The rule we hold ourselves to is simple and unforgiving. Before we change anything, we write down what we expect the change to do and, in the same breath, the exact result that would prove us wrong. A reason to disbelieve invented after the fact is not evidence. It is a story. Writing the falsifier first is what keeps a good-looking number from talking us into a change that did not actually help.
Then the measurement runs on a frozen copy of a real memory store, never on anything live, with one thing changed at a time. If the baseline does not reproduce first, the run is thrown out, because a number you cannot reproduce is not a result. And every store drifts a little on its own as it grows, so a change smaller than that natural wobble does not count. It has to clear the noise to be called a win.
A win is a win, and a revert is data
We do not hold a real improvement hostage to a bigger one that might exist later. A one percent gain that reproduces is shipped, then we go find the next one. Small wins are cumulative, and a lot of them is how the store gets sharper without a single risky leap.
The other half matters just as much. When a change fails to beat the baseline, it does not ship. When a change makes recall worse, it gets reverted, and the reason is written down instead of buried, so the same dead end does not get wandered into again half a year later. A revert is not an embarrassment here. It is information, and it is cheaper than shipping the regression to you.
No verdict is final
A memory layer is a living system with many parts feeding each other. Something that did nothing last month can start to matter the moment a layer beneath it is fixed, so we treat every verdict as provisional and re-open it rather than carving it in stone. Nothing stays live on reputation, and nothing is written off forever on one bad day. The store you run keeps being re-measured against itself.
Why measured beats trusted
A memory you merely trust is one you learn the truth about too late, in the session where it hands your agent the wrong thing and sounds completely sure of it. A memory you measure tells you, before it ever reaches your session, whether the right thing actually comes back when you ask. That is the whole difference, and it is why the boring work of benchmarking our own recall is the feature, not a chore behind it.
Where it stands
Curator is v0.3.0, a working eval-gated pre-release product dogfooded daily across the projects that build it. It answers to the same proof discipline as BenchProctor, pairs with TheAuditor for code ground truth and Warden as the agent that acts with memory in context, and shares a brain with Arbiter. Read ranked by truth, not recency for what the ranking optimizes, or a wall of text is not an answer for how we grade recall quality. Follow release preparation.
Was this useful?