Brainy Memory — Housekeeping

A memory brain keeps house. Four jobs run behind its doors, on the same Rust job runtime every other maintenance job in this engine uses, under the same three laws (see Clean Close):

1. A job never holds the doors. Work runs in installments of at most 50 ms and yields the moment foreground traffic moves.

2. A job leaves a durable progress marker. A killed process RESUMES.

3. A job is never silent — about anything it did. A job that finds nothing to do says nothing at all, which is the other half of the same law: a journal full of "nothing to do" is a journal nobody reads.

Not one of those is bounded by the SIZE of the store. That is deliberate and it is the single most important property on this page: a maintenance term that scales with the store rather than with the work costs microseconds on a test fixture and takes production down at scale.

They arm themselves on the first remember() — writing a memory is what makes a brain a memory brain — and on any housekeeping door. That same call declares the record shape, base fields and job fields together, so there is no manual declareProfile step before a store's first write and no way to end up with a sealed profile that refuses what the jobs write. There is no start button, and no configuration decides whether housekeeping happens.

Two durable records, and they answer different questions

A sweep leaves two things on the store, and neither can stand in for the other:

The marker is cleared when a fresh run starts — correct for what it is for, and fatal for a recurring sweep that mistook it for a watermark: every cycle would begin at the beginning of the store. For memory-consolidate that would mean one vector search per live memory every thirty minutes, forever, which is exactly the maintenance term that scales with the store rather than with the work. Both records are pinned, in both directions.

Decay: retrievability is never stored

A memory's retrievability — the probability you could still recall it — is the FSRS forgetting curve over stability and the time since lastRecalled || createdAt.

No job writes it. It is computed per candidate, at the moment a recall ranks. Three reasons, in the order they bite:

1. Exactness. A swept value is stale the moment it is written and wrong for the whole gap until the next sweep. A computed one is exact at the instant it ranks. Ranking on a value that is hours old is a wrong answer, not a slow one.

2. Cost. A sweep that writes a decayed value for every row is O(store) writes per cycle on a store nobody wrote to — segment-log growth, flush debt and compaction debt, forever, for a number nobody read in between.

3. One truth. Two writers of one number is how they come to disagree.

The consequence worth holding onto: ageing a memory changes what recall returns immediately, with no job having run at all.

What the decay job does write is the one stored signal decay's own inputs move: importance. A memory the world keeps reaching for should not decay out of reach, so a row's importance is set to what its counters have earned — one step per complete block of recalls, one per block of citations, capped below 1 so importance a writer set deliberately always outranks importance a job earned.

Earned, never incremented. An incrementing rule depends on how many times the sweep happened to look, which would make a store's importance values a record of the job's scheduling history rather than of the memory's use.

Episodes: structure only

Memories that happened together — same source and sourceConversationId, no more than the episode gap (30 minutes by default) of silence between them — are linked into an episode node: an Event of subtype memory-episode carrying when it started, when it ended, what source produced it and how many live members it has. Each member gets an in edge to it.

The engine writes no words. There is no language model inside this engine, so an episode's summary is a field the engine declares and a service fills. What the engine can say without a model — the dates, the source, the membership — is exactly what it writes, and the episode's own vector is the CENTROID of its members', which is a position it can compute rather than a sentence it would have to invent.

The walk is forward-only: a memory's place in the stream never changes, so a segment once decided is decided forever, and the durable cursor is a watermark rather than a position. Nothing is written on a row to mark it considered — the watermark IS the mark, which is what keeps the job's write count proportional to new memories rather than to the store.

A run is closed, and becomes an episode, when a later memory of the same source is more than the gap after it; or when the walk has seen the end of the store and the run has gone quiet; or when the run has spanned a day, the bound that stops a source which never pauses from holding one run open forever.

Consolidation: see the membership before the write

A memory written twice is one memory. The sweep searches each new memory's own vector against the live memories and treats anything at or above cosine 0.97 as the same memory: the newer row survives, the older is retired with reason duplicate and linked to the survivor by duplicateOf.

That threshold is one law, shared with the write-time duplicate check on remember(). Two numbers would mean a store whose duplicate policy depends on which door a row came in by.

// What would be consolidated, without writing anything:
const preview = await brain.memory.consolidate({ dryRun: true })
for (const cluster of preview.clusters) {
  console.log(cluster.survivorId, cluster.members)  // ids and cosines
}

// The same sweep, this time writing:
await brain.memory.consolidate()TS

The preview runs the identical code path at the identical threshold — a preview taken by a different path would be a preview of a program that does not exist.

Retirement is a demotion, never a deletion

Nothing in this engine deletes a memory on inference. A memory the engine judges redundant, superseded or spent does not leave the store; it leaves the default recall path.

await brain.memory.retire(id, {
  reason: 'superseded by the Monday ruling',
  supersededBy: replacementId,   // optional: a succession edge, one hop
})TS

After that call the row is still returned by get(id), still in history(), still in diff(), still in asOf() at every generation, still in every export. What changed is one field: recall's fence is "not retracted", and this row now is. Clearing it brings the memory back.

There is no forget(), and there will not be one.

The demotion itself lands in a single write, so a caller never waits for a job. What a retirement implies ELSEWHERE — its episode now has one fewer live member — is several writes, and several writes that must all land or all be resumable is what a marker-backed job is for. That is the memory-retirement pass, and it is also where the retention law is CHECKED rather than merely intended: every retired row it settles is read back by id, and a row the index calls retired that the store cannot produce is a loud fault naming the row.

Health

Every job reports a row: its state, when it last ran, how many rows it scanned, how many it WROTE, how much of its cycle budget it spent, and the runtime's own error and retry state.

for (const row of await brain.memory.jobs()) {
  console.log(row.job, row.state, row.rowsTouched, `${row.budgetUsedMs}/${row.budgetMs} ms`)
}TS

rowsTouched is the number to watch. On a store that is not changing it should be zero for every job, on every cycle, forever.

What a close does to them

A close does not finish background work. Every job is aborted at its next installment, what is already running is settled with a bound, and each marker is left exactly where it reached — so the next open resumes rather than restarting. Closing a brain that never used a memory door stands nothing down, because nothing was ever armed.